Kovalchuk assisted on Andy Greene's go-ahead goal 2:06 into the third period and then scored his 35th of the season on a wrist shot with 10:01 to play. He also had an assist on the goal by Patrick Elias that opened the scoring 1:43 into the game.
New Jersey won its second straight and moved within two points of Pittsburgh for the lead in the Atlantic Division.
The Penguins have lost two in a row after winning four straight. Their only goal came from Sidney Crosby less than a minute after the score from Elias.
Marc-Andre Fleury stopped 29-of-32 shots in the loss.
In addition to his three points, Kovalchuk had a chance to score on a penalty shot late in the second period, but he fired one directly into Fleury's pads.
The teams have met five times this season and New Jersey has won them all.
Exploding whale video goes viral on Internet
Gal Gadot cast as Wonder Woman for 'Batman vs. Superman'