Bayless connected on four 3-pointers and Andrea Bargnani finished with 19 points and eight rebounds for the Raptors.
Leandro Barbosa contributed to Toronto's win with 17 points off the bench.
Antawn Jamison scored 20 points for the Cavaliers, who had a three-game winning streak snapped. Ramon Sessions and Alonzo Gee each scored 16 points and Kyrie Irving supplied 14 points, seven boards and seven assists for the Cavs.
Celebrity Families of 2014 [PHOTOS]