With the first-round playoff confrontation deadlocked at 2-2, the series moves to New York Saturday for Game 5.
Turris took a drop pass from Jim O'Brien before firing the winning goal from the left faceoff circle. He beat Lundqvist to the glove side with the puck sailing just under the crossbar.
Ottawa overcame a 2-0 deficit to get the win.
Anton Stralman and Ryan Callahan scored on power plays for the Rangers in the opening period, with Stralman's goal coming 49 seconds into the game.
Craig Anderson blanked the Rangers the rest of the way while making 31 saves.
Lundqvist stopped 28-of-31 shots.
2014: NFL Cheerleaders [PHOTOS]
Michael Sam cut from the Dallas Cowboys