The Devils' David Clarkson forced the extra time with a goal following a strange bounce off the endboards with 48 seconds left in regulation.
Kovalchuk also scored in regulation, as did Zach Parise, helping New Jersey snap a three-game skid.
Anton Stralman, Brian Boyle and Michael Del Zotto scored and Biron finished with 26 saves for the Rangers.
Puzzle-maker slips 'Murdoch Is Evil' into Rupert Murdoch's Sunday Telegraph
Sign language interpreter at Mandela service called out as fake on Twitter