Horford (23 points, 11 rebounds) helped the Hawks wipe out a 10-point deficit over the final 6 1/2 minutes, sparking 20-9 run down the stretch.
Josh Smith hauled down 13 rebounds to go with 15 points for the Hawks.
J.J. Redick -- obtained in a trade from Orlando this week -- made his Bucks debut with 16 points, but it wasn't enough as Milwaukee tumbled for seventh time in eight contests.
Ersan Ilyasova contributed 19 points for the losers.
Ron Burgundy interviews Peyton Manning on SportsCenter
Man spent 15 hours in jail for plugging electric car into an outlet at a school