Stepan's deflection of a Rick Nash pass got past Braden Holtby with 6:25 left in the third period, breaking a 3-3 tie and giving the Rangers a crucial victory after falling behind 2-0 in the Eastern Conference quarterfinal series.
Henrik Lundqvist turned aside 28-of-31 shots for New York, which will attempt to even the series at home in Wednesday's Game 4.
Holtby finished with 26 stops for the Capitals.
Gal Gadot cast as Wonder Woman for 'Batman vs. Superman'
Couple mistakenly served bag of cash at McDonald's drive-thru