Davidson produced 14 second-chance points in the first half and battled Duke to a 29-29 tie at intermission.
Duke then scored the first 12 points of the second half to take control and advanced to 13-0 for the season.
Kelly scored 18 points, Quinn Cook added 15 and both Mason Plumlee and Tyler Thornton had 10 for the Blue Devils, who made 20-of-22 from the foul line.
Davidson (7-6) connected on just 17 field goals for the game and shot 29 percent. Jake Cohen had 19 points for the Wildcats.
It was the last non-conference game for Duke, which opens ACC action Saturday at home against Wake Forest.