The Rangers won their second in a row to get to 8-8 for the year. They are joined by Colorado as the only two teams in the NHL not to have played an overtime game this season.
Hagelin opened the scoring 4:16 into the game and his second goal, coming with 7:16 left in the middle period, gave the Rangers a 3-1 lead.
Ryan Murray cut the Columbus deficit in half less than a minute after Hagelin scored and New York had to protect a one-goal lead until Ryan McDonagh delivered an empty-net score with 54 seconds to go.
Cam Talbot, making the fifth start of his rookie season, improved to 4-1 by making 32 saves. He has allowed two goals in each of his appearances.
The loss went to Sergei Bobrovsky, who stopped 30 shots.
'Happy' fan videos make Pharrell cry [VIDEO]
Pregnant Mila Kunis wins 'Best Villain' at MTV Movie Awards