The Capitals had been the only NHL team without a point prior to Friday, but they managed one in the overtime loss after overcoming a 2-0 lead deficit in the third period.
New Jersey remained unbeaten in its three games.
The Devils appeared headed for a victory in regulation thanks to goals by Stephen Gionta with 6:05 left in the first period and Patrik Elias on the power play with a minute to go in the second.
Martin Brodeur, embarking on the 20th year of his record-setting career, ran his all-time regular-season win total to 659 by making 25 saves. Michal Neuvirth took the loss while stopping 30 shots.