LeMahieu's long balls came in his first two at-bats in the win Tuesday in New York City. The Yankees second baseman went 2 for 4 with two RBIs and two runs scored in the triumph.
Yankees starter Masahiro Tanaka allowed just three hits and two runs in six innings to pick up his first win of the season.
The Yankees (20-14) have now won four of their last five games and sit 3.5 games behind the first-place Rays (25-12) in the American League East.
Tanaka had two strikeouts and did not allow a hit in the first inning before LeMahieu stepped into the batter's box to leadoff the bottom of frame. He took a called strike from Rays starter Trevor Richards before he smacked an 0-1 changeup deep to left field for a 380-foot solo homer.
The ball traveled 101 mph before it sailed over the outfield fence, according to Statcast. Tanaka kept the Rays off the scoreboard through the next four innings.
LeMahieu returned to the plate in the third inning and smacked another solo shot to right field. That 352-foot blast left the field in 5.6 seconds.
Rays center fielder Kevin Kiermaier tied the score with a two-run homer off Tanaka in the top of the fifth frame. The Yankees took a 4-2 edge on a two-run double from Gio Urshela in the bottom of the sixth inning.
Urshela also scored on that play due to a throwing error from Rays shortstop Willy Adames to give New York a three-run lead.
Adames made up for his miscue with a solo homer in the top of the seventh inning, but the Rays couldn't rally over the final three innings.
Yankees closer Aroldis Chapman retired the Rays' final three batters in order to earn his first save of the season.
Richards allowed five hits and two runs in 4.2 innings for the Rays. Urshela went 2 for 4 with two RBIs and a run scored. Yankees first baseman Luke Voit also went 2 for 4 in the win.
The Yankees host the Rays in the final game of the series at 7:05 p.m. EDT Wednesday at Yankee Stadium.