Stepan's deflection of a Rick Nash pass got past Braden Holtby with 6:25 left in the third period, breaking a 3-3 tie and giving the Rangers a crucial victory after falling behind 2-0 in the Eastern Conference quarterfinal series.
Henrik Lundqvist turned aside 28-of-31 shots for New York, which will attempt to even the series at home in Wednesday's Game 4.
Holtby finished with 26 stops for the Capitals.
Putin thinks Obama would save him if he were drowning
Justin Bieber crashes Drake Bell's album release party