Trailing 3-1 heading into the sixth, the Orioles cut the deficit to a run when Nate McLouth led off with a walk from Yankees starter Phil Hughes (13-12) and Adam Jones and Matt Wieters followed with consecutive singles.
That brought up Reynolds, who slammed a 1-0 pitch deep into the left field seats for a 5-3 Baltimore lead. The first baseman also delivered a solo shot in fifth -- his second two-homer contest in the O's three-game set at Yankee Stadium.
Reynolds finished 3-for-5 with four RBI an three runs scored as second-place Baltimore took the rubber match of the series and moved to within two games of American League East-leading New York.
Randy Wolf (1-0) earned the win with 3 1/3 innings of one-run ball in relief of starter Chris Tillman.
Hughes surrendered five runs on eight hits over five full frames for the Yanks.
Chris Dickerson socked a two-run homer for New York.
Notable deaths of 2014 [PHOTOS]
UPI Sports Results for September 19, 2014