The Astros had grabbed their first lead of the game in the bottom of the eighth when J.D. Martinez delivered an RBI double to tie the score 2-2, and another run scored on the play when Reds first baseman Todd Frazier threw wildly to third.
The Reds came back in the next half-inning off Houston closer Francisco Cordero (0-2), who blew his second save opportunity in two games against Cincinnati.
Stubbs then came home on Jay Bruce's hit, and the National League Central-leading Reds continued their hot streak.
Sean Marshall (3-3) was credited with the win and Aroldis Chapman collected his 19th save to secure the victory.
Scott Moore supplied an RBI for Houston, which dropped its ninth straight.
Notable deaths of 2014 [PHOTOS]
Julius Randle breaks leg during NBA debut