Frazier's RBI single in the first inning put the Reds ahead to stay and Ludwick's run-scoring two-bagger in the third doubled the advantage. The pair each knocked in another run during a three-run eighth to help seal the victory.
Mat Latos (8-3) held the Astros to two runs on eight hits in a five-inning start to earn the victory.
Astros starter Wandy Rodriguez (7-9) was saddled with the loss, surrendering three runs on seven hits over six frames.
Chris Johnson went 4-for-5 with an RBI for Houston, which lost for the seventh straight time.