Horford (23 points, 11 rebounds) helped the Hawks wipe out a 10-point deficit over the final 6 1/2 minutes, sparking 20-9 run down the stretch.
Josh Smith hauled down 13 rebounds to go with 15 points for the Hawks.
J.J. Redick -- obtained in a trade from Orlando this week -- made his Bucks debut with 16 points, but it wasn't enough as Milwaukee tumbled for seventh time in eight contests.
Ersan Ilyasova contributed 19 points for the losers.
Scarlett Johansson steps out with fiance after pregnancy reveal
Dennis Rodman pledges to end trips to North Korea