The Astros had grabbed their first lead of the game in the bottom of the eighth when J.D. Martinez delivered an RBI double to tie the score 2-2, and another run scored on the play when Reds first baseman Todd Frazier threw wildly to third.
The Reds came back in the next half-inning off Houston closer Francisco Cordero (0-2), who blew his second save opportunity in two games against Cincinnati.
Stubbs then came home on Jay Bruce's hit, and the National League Central-leading Reds continued their hot streak.
Sean Marshall (3-3) was credited with the win and Aroldis Chapman collected his 19th save to secure the victory.
Scott Moore supplied an RBI for Houston, which dropped its ninth straight.