The Maple Leafs have won back-to-back games for the first time this season with James Reimer blanking the Capitals for the final 37:22 of the game.
Alex Ovechkin put Washington in front on the power play early in the second period with his second goal of the season, but Reimer took over and gave Toronto a chance to come from behind.
Kulemin scored on a tip-in 7:40 into the third period to tie it and Frattin produced the game-winner on a wrist shot.
James van Riemsdyk had a power-play goal for Toronto in the first period to create a 1-1 tie.
Reimer made 20 saves and Michal Neuvirth stopped 37 shots for the Capitals. Washington has just one victory in its first seven games.