The Braves led by a run heading into the ninth, but back-to-back singles followed by a sacrifice fly from Daniel Descalso allowed the Cardinals to force extra innings.
Neither side threatened in the 10th, but Cory Gearrin (0-1) hit Ryan Theriot with a pitch with one out in the 11th. Mark Hamilton then drew a two-out walk and Punto followed with his triple to right.
Trever Miller came on with one on and two out in the bottom of the 11th and struck out Nate McLough to end the game. Miller collected his first save and became the fifth St. Louis pitcher to save a game this season.
Eduardo Sanchez (1-0) worked the ninth and 10th innings for St. Louis to get the win.
Atlanta had solo homers from Jason Heyward in the first and McLough in the seventh.