Kreider netted an open-ice goal after catching the Capitals on a line change 7 minutes into the final frame, breaking a 1-1 tie, and Richards followed that with a close-in tally 1 1/2 minutes later.
Kreider added an assist for his first career two-point game and Artem Anisimov also found the net for the top-seeded Rangers, who grabbed a 1-0 lead in their seven-game Eastern Conference semifinal series against Washington.
Henrik Lundqvist wasn't sorely tested in the contest, turning aside 17-of-18 shots to earn the victory for New York.
Jason Chimera scored and Braden Holtby stopped 11-of-14 shots in the Caps' loss.
N.J. man wakes up from 10-hour sleep with knife in back
Puzzle-maker slips 'Murdoch Is Evil' into Rupert Murdoch's Sunday Telegraph