Dubinsky's power-play tally midway through the second period forged a 1-1 tie and Cam Atkinson's unassisted goal less than 5 minutes later on a backhanded shot from the high slot put Columbus ahead to stay.
Bobrovsky turned aside 24-of-25 shots as the Blue Jackets won a second straight following a four-game skid.
Michael Ryder supplied the only goal for Devils, who fell to 1-4-3 on the season.
Cory Schneider made 20 saves for New Jersey.
Jordana Brewster on Paul Walker: 'He was an enormous presence in my life'
Gal Gadot cast as Wonder Woman for 'Batman vs. Superman'