The Cubs seemed on their way to a win with two outs in the ninth inning and closer Kevin Gregg on the mound, but Starling Marte extended the contest with a clutch solo homer.
After a scoreless 10th, Rizzo and Alfonso Soriano led off the 11th with consecutive singles against Bryan Morris (4-3).
The Bucs then committed a key error with Luis Valbuena at the plate. After he missed on a sacrifice bunt attempt, catcher Russell Martin tried to pick off Rizzo at second but the ball sailed into left field, allowing both runners to advance.
After Valbuena was intentionally walked, pinch-hitter Navarro delivered a fly ball to right, allowing Rizzo to trot home with the game-winner.
Matt Guerrier (3-4) earned the decision with two innings of scoreless relief for the Cubs, who took the rubber match of a three-game series.
Jose Tabata went 2-for-3 with an RBI for Pittsburgh, which with the loss fell into a first-place tie with St. Louis atop the National League Central.