The comeback win evened the Eastern Conference semifinal series at 2-2. Pittsburgh will host Game 5 on Saturday and the series will return to Montreal on Monday. A Game 7, if needed, would be played in Pittsburgh next Wednesday.
Three goals were scored in the game's first 5:18, after which the contest settled into a tense duel during which the defending Stanley Cup champion Penguins tried to protect a lead that would have given them command of the series.
Montreal, however, thrilled their home fans by tying the game on a wraparound shot by Lapierre 2:07 into the third.
Gionta produced the winning goal 93 seconds later on what was meant to be a pass. He threw the puck in front of the net hoping a teammate would be able to gain control. The puck, however, bounced off the skate of Pittsburgh's Kris Letang and past goaltender Marc-Andre Fleury.
Montreal's Jaroslav Halak protected the lead by stopping Evgeni Malkin on a breakaway with 4 minutes to play.
Halak gave up goals to Maxime Talbot and Chris Kunitz in a span of 2 minutes early in the contest, but settled down to stop 33 shots. The goal by Talbot came on a breakaway and Kunitz scored when a pass from Sidney Crosby bounced off his skate and into the net.
Fleury allowed three goals on 25 shots.