Holland (1-0) baffled the Cardinals on two hits and a pair of walks while striking out seven in a dominating performance in which 76 of his 116 pitches were strikes.
Neftali Feliz recorded the final two outs to wrap up the victory for Texas, which shut down the Cardinals' offense after yielding 16 runs in Saturday's Game 3 loss.
Mike Napoli cracked a three-run homer -- his second of the World Series and third of the post-season -- and Josh Hamilton added an RBI double for the Rangers, who will seek to a grab a 3-2 lead in Monday's Game 5 in Arlington.
Edwin Jackson (0-1) absorbed the loss for the Cardinals, allowing three runs on three hits over 5 1/3 innings.
2014: The Year in Music [PHOTOS]