The game had been tied since the sixth, when the Reds completed a comeback from a four-run deficit. That rally began with a three-run fifth that included a two-run double by Brandon Phillips.
Arredondo (0-3) gave up a double to J.J. Hardy to open the 11th but worked out of that inning and was sent out again in the 12th.
Lee quickly ended things by sending a 1-2 pitch over the wall in left for his fifth homer of the season. That made a winner out of Mike Gonzalez (1-1), who picked up his 16th victory in a nine-year career.
Gonzalez almost gave up the go-ahead run in the top of the 11th but Joey Votto ended the inning when he was thrown out at home trying to score from first on a double by Jay Bruce.
Baltimore's Adam Jones hit a homer in the third while the Orioles were building a 4-0 lead.
'SNL': 'Anchorman 2' cast, One Direction sing 'Afternoon Delight' [VIDEO]
Exploding whale video goes viral on Internet