Rizzo smashed two-run home runs in both the first and third innings and four Chicago pitchers made the lead stand up.
Feldman (1-3) gave up two runs on seven hits with two walks in 6 1/3 innings. Kevin Gregg came on to work a perfect ninth for his second save.
Chicago has won two games in a row for the first time this season. Miami has yet to do so and is the only team in the majors without consecutive victories.
Dave Sappelt scored ahead of Rizzo on both his homers, reaching base on a single to open the game and then delivering an infield single in the third.
Wade LeBlanc (0-4) took the loss, allowing both homers and six other hits in six-plus innings.
Putin thinks Obama would save him if he were drowning
The Giudices celebrate Easter as they prepare for jail