Zach Randolph broke a tie game and put Memphis in front for good by hitting two free throws with 3 minutes remaining and Conley's long-range shot gave the Grizzlies an 83-77 lead.
Memphis held on to win its fifth straight and has the best record in the NBA (35-18) for a team not leading its division.
Randolph and Conley both scored 17 points and Randolph claimed 18 rebounds.
Toronto trailed by 11 at the half after managing just 32 points in the opening two periods. Alan Anderson had 19 points off the bench to lead the Raptors, who had captured five straight.
Celebrity Families of 2014 [PHOTOS]