The Cardinals had a five-game winning streak snapped and they fell back into a tie with Pittsburgh for the lead in the National League Central.
Tyler Thornburg (2-1) gave up two runs on three hits in six innings while striking out six to help salvage the last of a three-game series. Jim Henderson came on in the ninth to protect a three-run lead and allowed a solo homer by Matt Adams on the way to his 25th save.
Halton's homer in the fourth came after Carlos Gomez had singled to open the inning and gave Milwaukee a 4-0 cushion.
The loss went to Joe Kelly (8-4), who surrendered four runs on seven hits in five innings.
Celebrity Breakups and divorces of 2014 [PHOTOS]