The score was tied 2-2 in the fourth when Joyce smacked the first pitch he saw from Orioles starter Chris Tillman onto the porch in right field, putting the Rays ahead to stay.
Evan Longoria ripped a solo homer for the Rays, who came into the contest trailing American League East-leading Boston by one game.
David Price (7-5) surrendered 10 hits and walked two in five innings but pitched well enough to earn the decision, yielding two runs while striking out six.
Matt Wieters went 3-for-5 with a solo homer and two RBI for the third-place Orioles, who started the night three games out of an AL wild card spot.
Celebrity Families of 2014 [PHOTOS]