Almonte was called up from Class AAA Scranton/Wilkes Barre earlier this week and had appeared as a pinch-hitter in two games before getting the start in left field against the Rays.
He went 3-for-4 and his sixth-inning solo homer gave the Yankees a 5-2 lead.
David Phelps (5-4) gave up eight hits over 5 2/3 innings, but limited the Rays to two runs to pick up the win. The Yankees snapped a two-game skid and picked up their 40th victory of the season.
The loss was charged to Roberto Hernandez (4-8), who gave up five runs on nine hits in seven innings.
Beyonce flaunts bikini body, Blue Ivy in vacation pics
Gal Gadot cast as Wonder Woman for 'Batman vs. Superman'