Dubinsky's power-play tally midway through the second period forged a 1-1 tie and Cam Atkinson's unassisted goal less than 5 minutes later on a backhanded shot from the high slot put Columbus ahead to stay.
Bobrovsky turned aside 24-of-25 shots as the Blue Jackets won a second straight following a four-game skid.
Michael Ryder supplied the only goal for Devils, who fell to 1-4-3 on the season.
Cory Schneider made 20 saves for New Jersey.
Attkisson leaves CBS News, reportedly over network's 'liberal bias'
Astronomers offer more expansive view of universe