Kovalchuk assisted on Andy Greene's go-ahead goal 2:06 into the third period and then scored his 35th of the season on a wrist shot with 10:01 to play. He also had an assist on the goal by Patrick Elias that opened the scoring 1:43 into the game.
New Jersey won its second straight and moved within two points of Pittsburgh for the lead in the Atlantic Division.
The Penguins have lost two in a row after winning four straight. Their only goal came from Sidney Crosby less than a minute after the score from Elias.
Marc-Andre Fleury stopped 29-of-32 shots in the loss.
In addition to his three points, Kovalchuk had a chance to score on a penalty shot late in the second period, but he fired one directly into Fleury's pads.
The teams have met five times this season and New Jersey has won them all.