After spotting the Capitals a quick 1-0 lead in the first period, the Rangers pulled even in the middle frame as Hagelin scored the equalizer on assists from Rick Nash and Ryan McDonagh.
Washington's Karl Alzner was whistled for tripping Brad Richards at 4:18 of the third period, putting the Rangers on a power play. Stepan converted a precise pass from Michael Del Zotto for the go-ahead tally.
Henrik Lundqvist stopped 27-of-28 shots for the Rangers, rebounded from Thursday's 4-3 overtime loss to the New York Islanders.
John Carlson scored and Braden Holtby turned aside 38-of-40 shots for Capitals, whose three-game winning streak was broken.
Aaron Carter is still in love with Hilary Duff
Ray Liotta sues skin care company over use of likeness