Johansen erased a 3-2 deficit on a score with 4:02 remaining in the third period.
Skille then notched the game-winner when he received a behind-the-back pass from Artem Anisimov and fired a shot past Hurricanes goalie Justin Peters to the short side.
Mike McKenna preserved the victory with a save on Jeff Skinner in the closing seconds, finishing with 30 stops.
Anisimov also scored and Corey Tropp added a goal as the Blue Jackets took a second straight win.
Skinner had two goals and Alexander Semin supplied a goal and an assist as Carolina fell for the sixth time in seven games.
Peters logged a season-high 47 saves.