Dubinsky's power-play tally midway through the second period forged a 1-1 tie and Cam Atkinson's unassisted goal less than 5 minutes later on a backhanded shot from the high slot put Columbus ahead to stay.
Bobrovsky turned aside 24-of-25 shots as the Blue Jackets won a second straight following a four-game skid.
Michael Ryder supplied the only goal for Devils, who fell to 1-4-3 on the season.
Cory Schneider made 20 saves for New Jersey.
Benedict Cumberbatch's dramatic reading of R. Kelly lyrics is just what you need
Beyonce flaunts bikini body, Blue Ivy in vacation pics