Foster's 1-yard scoring run in the third quarter put the Texans ahead 16-7 and Houston used a tough defense to limit Cincinnati to just 207 total yards of offense.
He also made eight catches for 34 yards, becoming the first player in NFL history to rush for 100 or more yards in his first three career playoff games.
Matt Schaub completed 29-of-38 passes for 261 yards with an interception and Shayne Graham went 4-for-4 on field goal attempts as the Texans ended the Bengals' season in the wild card round for the second straight season.
Owen Daniels compiled 91 receiving yards on nine catches for Houston, which goes on to play second-seeded New England next week.
Bengals quarterback Andy Dalton was held to just 127 passing yards with an interception on 14-of-30 efficiency.
Cincinnati remained winless in the post-season since 1991.
Exploding whale video goes viral on Internet
Couple mistakenly served bag of cash at McDonald's drive-thru