The Rangers won their second in a row to get to 8-8 for the year. They are joined by Colorado as the only two teams in the NHL not to have played an overtime game this season.
Hagelin opened the scoring 4:16 into the game and his second goal, coming with 7:16 left in the middle period, gave the Rangers a 3-1 lead.
Ryan Murray cut the Columbus deficit in half less than a minute after Hagelin scored and New York had to protect a one-goal lead until Ryan McDonagh delivered an empty-net score with 54 seconds to go.
Cam Talbot, making the fifth start of his rookie season, improved to 4-1 by making 32 saves. He has allowed two goals in each of his appearances.
The loss went to Sergei Bobrovsky, who stopped 30 shots.
Susan Sarandon 'very excited' about daughter's pregnancy
Dennis Rodman pledges to end trips to North Korea