Joyce's two-run double capped a four-run third inning that put the Rays ahead to stay. He added a solo homer in the eighth.
Ryan Roberts also went deep for Tampa Bay, which had lost the first two games of a four-game set against the Blue Jays.
Starter Jeff Niemann, out since May 14 when his right leg was fractured by a line drive off the bat of Toronto's Adam Lind, returned from the disabled list but could go only 3 1/2 scoreless innings before leaving with shoulder tightness.
Davis (2-0) responded by hurling 2 1/3 shutout relief frames to earn the victory and Fernando Rodney collected his 40th save, recording the final five outs for the Rays.
Toronto's Henderson Alvarez (7-12) was touched for four runs on five hits over five full innings, absorbing the loss.
Edwin Encarnacion belted his 36th homer -- a two run shot -- for the Blue Jays.
Duggar sisters unveil Christian dating rules in new book
Attkisson leaves CBS News, reportedly over network's 'liberal bias'