Freddie Freeman went 4-for-5 with two solo homers and Dansby Swanson added another solo homer for Atlanta (1-5), which lost its fourth in a row.
Felipe Rivero (1-0) pitched the 10th for Pittsburgh (3-2) to pick up the win. Jose Rameriz (0-1) pitched the 10th for Atlanta.
Braves starter Julio Teheran gave up two runs -- both unearned -- on six hits, with four strikeouts and one walk over seven innings.
Pittsburgh starter Gerrit Cole allowed runners in each of his six innings. He allowed three runs on eight hits, including two homers, with four strikeouts and two walks.
Cole struggled from the outset. He got Ender Inciarte to fly out on his first pitch, but on his third Swanson hit a homer to left for a 1-0 Braves lead. Later in the first, Nick Markakis' two-out RBI single drove in Freeman to make it 2-0.
Teheran, meanwhile, pitched to just one batter over the minimum through three innings. In the fourth, Starling Marte singled and moved to second on Freeman's error at first after Teheran's pickoff attempt caught Marte off the bag. Two outs later, Josh Bell doubled home Marte. Jordy Mercer followed with an RBI single to tie it 2-2.
Freeman hit a one-out, first-pitch homer to left in the fifth to give Atlanta a 3-2 lead.
Cole worked out of another tight spot in the sixth, striking out Inciarte with two out and runners on second and third. The strikeout was completed on Cole's 98th and last pitch.
Freeman hit his second homer, to right, off of Pittsburgh reliever Antonio Bastardo with one out in the seventh for a 4-2 lead.
The Pirates pulled to within 4-3 in the eighth when reliever Arodys Vizcaino hit Josh Harrison with one out, Adam Frazier singled to left, and left fielder Jace Peterson misplayed Marte's single, allowing Harrison to score.
Gregory Polanco led off the ninth with a single to left off of reliever Jim Johnson and advanced to second on a passed ball. Josh Bell walked, with Alen Hanson coming in as a pinch runner. After Mercer popped out, pinch-hitter David Freese walked to load the bases for pinch-hitter Francisco Cervelli.
Cervelli's fielder's choice -- broken up at second by Freese -- allowed Polanco to score for a 4-4 tie.
NOTES: 2B Phil Gosselin made his first start with the Pirates. ... 3B David Freese was out of the Pittsburgh starting lineup for the second day in a row after taking a ground ball to the collarbone on Friday but entered the game in the ninth. ... LF Matt Kemp remained out of the Atlanta lineup after leaving Friday's game because of right hamstring tightness. ... This was the sixth game of a season-opening eight-game, 10-day road trip for the Braves.