Kelly (3-5) -- who faces the possibility of being moved out of the starting rotation when Jaime Garcia returns from a shoulder sprain -- made the start a solid one, scattering six hits and striking out six.
Kelly allowed a two-run homer by Miguel Montero in the seventh inning that cut the St. Louis lead to 3-2. The Cardinals responded with two runs in the bottom of the seventh and three more in the eighth to put the game out of reach.
Matt Holliday and Jon Jay homered for the Cardinals and pinch-hitter Matt Carpenter drove in a pair.
Ian Kennedy (10-10) took the loss, allowing five runs -- three earned -- on six hits in six innings.
Arizona lost for the fourth time in its last six games.