Joyce's sacrifice fly in the third inning put the Rays ahead 3-1 but the Orioles halved the deficit with a run in the seventh.
Clinging to a one-run lead, the Rays broke the game open in the ninth when they loaded the bases with none out against T.J. McFarland and Joyce followed by lining a 1-1 pitch down the right field line for a two-run double.
Wil Myers completed the outburst with a two-run single and second-place Tampa Bay -- which came into the contest trailing American League East-leading Boston by one game -- recorded a third straight win.
Alex Cobb (8-2) yielded two runs (one earned) on three hits, overcoming four walks and striking out five to get the victory.
Fernando Rodney wrapped up the triumph with his 29th save.
Orioles starter Miguel Gonzalez (8-6) was tagged with the loss, surrendering three runs (two earned) on eight hits in a 5 2/3-inning effort.
Matt Wieters cracked a two-run homer for Baltimore, which fell 4 1/2 games out of an AL wild card spot.