The score was tied 2-2 in the fourth when Joyce smacked the first pitch he saw from Orioles starter Chris Tillman onto the porch in right field, putting the Rays ahead to stay.
Evan Longoria ripped a solo homer for the Rays, who came into the contest trailing American League East-leading Boston by one game.
David Price (7-5) surrendered 10 hits and walked two in five innings but pitched well enough to earn the decision, yielding two runs while striking out six.
Matt Wieters went 3-for-5 with a solo homer and two RBI for the third-place Orioles, who started the night three games out of an AL wild card spot.
Gal Gadot cast as Wonder Woman for 'Batman vs. Superman'
Beyonce flaunts bikini body, Blue Ivy in vacation pics