Brian McCann got a one-out single off Jeff Karstens (3-3) in the top of the 11th. Wilkin Ramirez came in to run for him and Conrad then golfed a 1-2 pitch over the right-center field fence for a two-run Braves lead.
Craig Kimbrel made the advantage hold up in the bottom of the frame, retiring the Pirates in order, and George Sherrill (1-0) earned the win thanks to a scoreless 10th for Atlanta, which swept a two-game series in Pittsburgh.
Eric Hinkse hit a solo homer and Chipper Jones added an RBI for the Braves.
Steve Pearce went 3-for-5 with a pair of RBI, including a game-tying solo homer in the seventh, in Pittsburgh's loss.