Miller (4-2) was touched for two runs on six hits with a pair of walks and five strikeouts for the Cardinals, who have won their past two. Matt Holliday and Peter Bourjos each drove in a run.
Aaron Harang (3-3) gave up four runs -- two earned -- on nine hits with a walk and six strikeouts over six innings for the Braves, who have dropped their past seven games. Ramiro Pena hit a solo home run in the loss.
The game stayed scoreless until the fifth when the Cardinals went on top. Bourjos led off the frame with a single and moved to second as Andrelton Simmons misplayed a Mark Ellis grounder. A sac bunt from Miller moved the runners up and Carpenter stroked a double to right-center to score both runners. Holliday added an RBI single later in the frame for a 3-0 lead.
The Cardinals loaded the bases later in the frame and appeared to score another run as Yadier Molina reached base when his grounder back to the mound deflected off Harang. Chris Johnson grabbed the ball and reached out his leg to tag third, but the runner was called safe and a man crossed the plate. The play was challenged, though, and reversed for the third out of the inning.
Atlanta cut its deficit in half in the bottom of the sixth. Justin Upton led off with a walk and Freddie Freeman followed with a single. Evan Gattis then singled to push one run across, and B.J. Upton's sac fly off Pat Neshek made it a 4-2 game.
Pena's solo home run in the seventh made it a one-run game, but the Braves stranded a man on second in the inning.
After Carlos Martinez pitched a 1-2-3 eighth inning, Trevor Rosenthal got into some trouble in the ninth.
Simmons singled to start and moved to second on a sac bunt. After Pena flied out, Jason Heyward walked to put the go-ahead run on base, but Rosenthal got Justin Upton to look at strike three for his ninth save of the season.