The Reds were trailing 2-1 in the sixth when John Axford (6-7), in relief of Brewers' starter Wily Peralta, surrendered a score-tying solo blast to Ludwick. After Jack Hannahan singled, Cozart (three RBI) smacked a 3-2 pitch into the left field stands for the decisive score.
Chris Heisey and Joey Votto each added a bases-empty home run for Cincinnati, which extended its lead for the second National League wild-card spot to 7 1/2 games over Arizona and pulled to with three games of NL Central-leading Pittsburgh.
Bronson Arroyo (13-9) yielded three runs on seven hits over seven innings, walking none and striking out six to notch his fourth win in four starts.
J.J. Hoover tossed a scoreless eighth and Aroldis Chapman secured the win with his 33rd save.
Caleb Gindl homered for Milwaukee.