Kelly (3-3) surrendered one run on six hits, walking two and striking out one as the Cardinals reeled off four double plays en route to the victory.
The right-hander also provided a run-scoring groundout in the fifth for a 2-0 St. Louis lead.
Carlos Beltran slammed a solo homer in the eighth and Matt Adams added a two-run blast later in the inning for second-place St. Louis, which rebounded from a 3-2 loss in Monday's series opener and remained two games behind National League Central-leading Pittsburgh.
Clayton Kershaw (10-7) was hung with the loss after yielding two runs on six hits over six full innings as the Dodgers saw their 15-game road winning streak -- the second-longest in NL history -- snapped with the defeat.
Adrian Gonzalez provided the only RBI for NL West-leading Los Angeles.