Allen, who joined the Rays last week after being waived by the Oakland Athletics, was sent in to hit for Jose Molina with B.J. Upton on first and one out. Los Angeles reliever Jordan Walden (0-1) had the count at 2-2 when Allen connected for his home run.
Brandon Gomes (1-1) pitched the ninth and picked up the win.
The first two Tampa Bay batters had hits and accounted for a run and the 1-0 lead held until the fifth when Los Angeles got a leadoff home run by Mark Trumbo. It was his third of the season.
Trumbo doubled in another run in the sixth and scored on a Vernon Wells base hit that put the Angels up 3-1. The Rays kept within striking distance with an RBI-triple by Matt Joyce in the bottom of the inning. His hit drove in Luke Scott who walked with two outs.