The Hawks took an 18-point lead into the fourth quarter and held on to bounce back from a surprisingly one-sided home loss on Monday against Dallas.
Teague scored 27 points and had 11 assists. Horford added 15 rebounds to his 26 points. Atlanta held the Bucks to 37 percent shooting.
Larry Saunders scored 19 points with 14 rebounds for Milwaukee, which had won two in a row. Brandon Jennings had 21 points, 19 of them in the second half.
Monta Ellis, who averages 19 points a game for the Bucks and is their leading scorer, hit just 2-of-14 from the field and wound up with five points.
Exploding whale video goes viral on Internet
Couple mistakenly served bag of cash at McDonald's drive-thru