The Phillies won for the fifth time in six games, with Victorino getting two hits, scoring a run, walking once and stealing two bases.
Feliz singled to open the ninth off David Weathers (1-2), who had come on in the eighth as Cincinnati's fourth pitcher of the game.
Paul Bako popped out on an attempted sacrifice bunt and pinch-hitter Eric Bruntlett was retired on a fly ball to right.
Given a good chance to get out of the ninth and force extra innings, Weathers walked Jimmy Rollins and then gave up the single to center by Victorino.
After Cincinnati took a 2-0 lead in the fourth, Victorino scored in the bottom half of the inning on a single by Ryan Howard. Jayson Werth tied the score in the sixth with his 19th homer of the season.
The win went to Ryan Madson (3-4), who worked two innings of perfect relief.