The late comeback made a winner of David Robertson (1-0), who tossed a scoreless bottom of the eighth.
New York starter Phil Hughes pitched well in an eight-inning no-decision, holding the Rays to two runs on six hits while striking out six.
David Price (0-2) absorbed the hard-luck loss for Tampa Bay. He remained winless on the season after surrendering three runs on eight hits over eight full frames.
Jose Molina provided an RBI for the Rays, whose four-game winning streak came to an end.