Holliday then led off the sixth by lining a 3-1 pitch into the right-field bleachers, putting National League wild card-contending St. Louis ahead for good.
Yadier Molina added a sacrifice fly before a 35-minute rain delay and the Cardinals hung on to take first of a three-game series, winning for the sixth time in seven games overall.
Lohse (14-2) allowed a pair of runs on five hits over five full frames to earn his 14th win, one short of a career high, and Jason Motte recorded the final five outs for his 31st save.
Pittsburgh's A.J. Burnett (15-5) was saddled with the loss, yielding four runs (three earned) on seven hits through 5 2/3 frames.