The Thrashers moved into a tie with Carolina for first place in the Southeast Division.
Atlanta was fortunate to get into overtime, coming up with a goal from Marian Hossa with just 18.1 seconds remaining when the Devils botched an attempt to clear the puck.
Both sides produced one goal in the first three rounds of the shootout and both delivered a goal in the fourth round. But nobody else could score until Dupuis got the best of Martin Brodeur with a wrist shot that sailed over the goaltender's glove.
Arron Asham then missed the net for New Jersey to end the game.
It was the third straight win for the Thrashers, who also got regulation goals from Todd White and Ilya Kovalchuk. Johan Hedberg stopped 30 shots and then turned away eight of the 10 skaters in the shootout.
New Jersey had won its previous two games.
2014: The Year in Music [PHOTOS]