Johnson's two-run blast in the fifth gave Tampa Bay a 2-1 advantage and he produced another home run in the seventh to put the Rays in front 4-3.
The Rays grabbed two out of three in the series and moved a half game in front of New York in the battle for the American League East lead. Boston is 6 1/2 games out.
Tampa Bay and the Yankees will meet four more times, beginning Monday in New York.
The win went to Chad Qualls (2-4), who recorded the last two outs in the seventh inning. It took some additional clutch relief pitching, however, to preserve the win.
New York had runners on first and second with one away in the eighth before Grant Balfour struck out Austin Kearns and got Colin Curtis on a grounder to second to end the threat.
Rafael Soriano came on in the ninth and quickly got two outs. Mark Teixeira then reached on an eight-pitch walk, but Soriano struck out Alex Rodriguez swinging to end the game.
It was Soriano's 43rd save.
Curtis Granderson hit a two-run homer for the Yankees in the seventh to give New York a 3-2 lead.
Prior to meeting again next week, Tampa Bay will host the Los Angeles Angels for three games and New York will play three times in Baltimore.