The Islanders were trailing 4-3 when Mark Streit notched his second goal of the contest 4 1/2 minutes into the final frame, producing a 4-4 deadlock.
Then at the 10:11 mark, Tavares got control the puck in the slot and weaved through traffic, launching a shot at Marc-Andre Fleury.
The Penguins goalie made a sprawling toe save, but the rebound came back to Tavares, who shoveled it home for the decisive score.
Casey Cizikas added an insurance tally to go along with two assists and Streit also provided a helper, finishing with three points as the Islanders rebounded from an overtime loss in Game 3 to forge a 2-2 tie in their Eastern Conference quarterfinal series.
Evgeni Nabokov turned aside 27-of-31 shots in the victory.
Fleury finished with 18 saves in the loss.