New York moved a full game in front of idle Baltimore at the top of the American League East. The Yankees will face Oakland in a weekend series and Baltimore will visit Boston for three games.
Ichiro Suzuki, who had seven hits during New York's sweep of a doubleheader on Wednesday, had three RBI on Thursday.
Suzuki hit a solo homer in the third, but New York faced a 2-1 deficit heading into the fourth. Suzuki then put the Yankees in front for good with a two-run double and he later scored on Swisher's grand slam that gave New York an 8-2 lead.
Phil Hughes (16-12) worked only five innings to get the win, striking out nine while giving up four runs on four hits. After Toronto closed within three runs in the eighth, David Robertson came on to strike out the side in the ninth and claim his second save.
Aaron Laffey (3-6) was touched for five runs on two hits and five walks in three-plus innings to suffer the loss.