Kovalchuk delivered his goal in the first round and Johan Hedberg was perfect for the Devils in the shootout after stopping 25 shots in regulation and overtime.
New Jersey won its second straight on the heels of a three-game losing streak. Calgary has lost three in a row and six of its last seven.
David Clarkson scored for the Devils in the first period and it remained a 1-0 game until Rene Bourque produced his 10th goal of the season 1:18 into the third.
Calgary's Henrik Karlsson took part in the first shootout of his career after making 20 saves over the first 65 minutes.
It was the only meeting between the teams this year. Calgary last won in New Jersey five years ago.
2014 amfAR Inspiration Gala [PHOTOS]
Julius Randle breaks leg during NBA debut