Bishop had faced two players in the shootout, with one shot wide and Bishop saving the other, before Silfverberg beat New Jersey goaltender Martin Brodeur for the lead.
Bishop forced another shot wide in sealing the win.
The Devils took a 1-0 lead 19 seconds into the game when Stephen Gionta got the puck by Bishop on the Devils' second shot of the game. But Bishop, making his third start of the season, had 29 saves the rest of the way.
Ottawa tied the score midway through the third period with a backhander from Daniel Alfredsson.
Brodeur ended with 30 saves and was first star of the game, despite taking the loss. Bishop was second star.
Ron Burgundy interviews Peyton Manning on SportsCenter
Man spent 15 hours in jail for plugging electric car into an outlet at a school