Bay, who had a pair of doubles, drove in David Ortiz to break a 3-3 tie.
Jonathan Papelbon closed out the game by striking out three consecutive batters in the ninth after the first two hitters had reached base safely. Papelbon earned his eighth save while Ramon Ramirez (3-0) got the win in relief in the rubber match of the three-game series.
Brad Shouse (1-1) took the loss after giving up the Ortiz double that turned into the winning run.
Carl Crawford led the Rays attack with two hits and an RBI. Crawford scored the first run of the game in the first inning when Pat Burrell singled off Red Sox starter Josh Beckett.
Boston went ahead 3-1 in the fourth on an RBI double by Jason Varitek and an RBI single from Nick Green.
Tampa came back to make it 3-2 in the fifth when Crawford singled home B.J. Upton, and then tied the game in the sixth on a sacrifice fly by Jason Barrett.
Celebrity Breakups and divorces of 2014 [PHOTOS]