The Maple Leafs suffered another disappointing setback, having won just once in their 11 outings this season.
Toronto tied it with only 38 seconds remaining regulation on a shorthanded goal by Mikhail Grabovski.
Connolly, however, produced his second goal of the game on the power play, taking advantage of Matt Stajan's absence for interference.
Connolly's first goal, which came on a wrist shot midway through the second period, gave Columbus a 2-1 lead.
Ryan Miller made 33 saves for the victory. Jonas Gustavsson stopped 25-of-28 shots in defeat.
Megyn Kelly: Santa Claus and Jesus are both white men
Penelope Cruz named 2013's 'Best Body' by Fitness magazine