The division-leading Braves have lost four out of five and have scored more than three runs just twice in their last 10 outings.
Both of the Atlanta hits came in the fifth, but B.J. Upton was thrown out at home during that inning to help Peralta get out of trouble.
Milwaukee pitching allowed six walks, two of them given up by Jim Henderson in the ninth. With the tying runs aboard, Henderson got Dan Uggla on a fly ball to center that ended the game.
Peralta (5-8) worked seven innings to get the win. It was Henderson's 10th save.
Milwaukee scored in the first on a sacrifice fly by Aramis Ramirez and in the fifth on Jean Segura's 11th home run of the season.
Both runs were charged to Julio Teheran (5-4), who gave up eight hits over 6 1/3.