The Capitals had been the only NHL team without a point prior to Friday, but they managed one in the overtime loss after overcoming a 2-0 lead deficit in the third period.
New Jersey remained unbeaten in its three games.
The Devils appeared headed for a victory in regulation thanks to goals by Stephen Gionta with 6:05 left in the first period and Patrik Elias on the power play with a minute to go in the second.
Martin Brodeur, embarking on the 20th year of his record-setting career, ran his all-time regular-season win total to 659 by making 25 saves. Michal Neuvirth took the loss while stopping 30 shots.
Exploding whale video goes viral on Internet
Campus cop fatally shoots Texas student during traffic stop