They tied in it seventh on a two-run homer by Michael McKenry and took the lead in next frame when Walker's routine fly ball to left fell in after New York outfielders Kirk Nieuwenhuis and Mike Baxter collided while setting up for the catch.
Walker went all the way to third on the miscue and scored on a Clint Barmes sacrifice fly, just beating the tag from catcher Mike Nickeas.
The rally made a winner of Jared Hughes (1-0), who tossed a scoreless top of the eighth, and Joel Hanrahan picked up his 10th save as Pittsburgh took the first of a three-game series at PNC Park.
New York's Jon Rauch (3-3) was saddled with the loss on the unearned run.
Justin Turner drove in two runs for the Mets.