It was Kovalchuk's first goal in eight games and fourth of the season.
The winning score came on the power play off a feed from Henrik Tallinder. Kovalchuk sent a one-timer past Devan Dubnyk.
Edmonton had a 3-1 lead late in the second period but Brian Rolston scored his first goal of the season with 3:08 to go in the second and Mattias Tedenby deadlocked the game on the power play with 9 minutes left in regulation.
New Jersey had lost its first seven home contests, two of the setbacks coming in overtime.
Martin Brodeur, who had missed two games with a bruised elbow, made 21 saves for the victory.
Edmonton has lost three in a row.