New York forced extra innings in the ninth thanks to a solo homer from Francisco Cervelli off J.J. Putz and the Yankees then wasted a scoring chance in the 11th.
David Phelps (0-1) took the mound for the Yankees in the 12th and gave up a leadoff double to Gerardo Para.
Para took third on a fly out and Paul Goldschmidt reached first on the rarely called catcher's interference.
Phelps hit Miguel Montero with a pitch to load the bases and Cody Ross singled to drive in the go-ahead run. With the bases still loaded, Chavez delivered a double to center that brought home all three runners.
Tony Sipp came on in the bottom of the inning and retired the side in order in a non-save situation.
Heath Bell (1-0) earned the win by working the 11th, during which New York put two runners on without scoring.
New York had a three-game winning streak snapped.