Stepan's deflection of a Rick Nash pass got past Braden Holtby with 6:25 left in the third period, breaking a 3-3 tie and giving the Rangers a crucial victory after falling behind 2-0 in the Eastern Conference quarterfinal series.
Henrik Lundqvist turned aside 28-of-31 shots for New York, which will attempt to even the series at home in Wednesday's Game 4.
Holtby finished with 26 stops for the Capitals.
Video of Victoria’s Secret models trying to 'twerk' hits Instagram
Costly malfunction causes beer flood at Boston-area brewery