Calvert, Artem Anisimov and David Savard all delivered a goal during the opening 11:10 to give Columbus a 3-0 led, but the Rangers fought back to within one midway through the third period.
Ryan Johansen's 11th goal of the season midway through the final stanza wrapped up the victory, the second in a row for the Blue Jackets.
New York's goals came from Dominic Moore and Dan Girardi, but the Rangers suffered their fourth straight setback.
Both sides made goaltender changes. New York's Henrik Lundqvist was removed in favor of Cam Talbot after giving up three goals on 10 shots. Curtis McElhinney and Mike McKenna were both given work in place of the injured Sergei Bobrovsky. McElhinney and McKenna gave up a goal each.