Horford (23 points, 11 rebounds) helped the Hawks wipe out a 10-point deficit over the final 6 1/2 minutes, sparking 20-9 run down the stretch.
Josh Smith hauled down 13 rebounds to go with 15 points for the Hawks.
J.J. Redick -- obtained in a trade from Orlando this week -- made his Bucks debut with 16 points, but it wasn't enough as Milwaukee tumbled for seventh time in eight contests.
Ersan Ilyasova contributed 19 points for the losers.
Celebrity Couples of 2014 [PHOTOS]