Connolly's tally 3 seconds after a penalty to the Blues' Vladimir Sobotka had expired came on a tip-in of a Radko Gudas shot 8:37 into the third period, breaking a 2-2 tie.
Valtteri Filppula added an insurance goal for the Bolts with 3:59 remaining.
Steven Stamkos also scored and Ben Bishop made 28 saves as Tampa Bay won a second straight and fifth in six games.
Jaden Schwartz and Alex Pietrangelo each scored for St. Louis.
Jaroslav Halak made 21 saves.