Willingham's two-run shot in the third put Oakland ahead for good at 2-1 and Matsui's solo homer in the seventh gave the A's a much-needed insurance run.
Jemile Weeks went 2-for-4 with an RBI single for Oakland, which rebounded from Saturday's 17-7 loss and evened its three-game series with the Yankees at one win apiece.
Rich Harden (2-1) earned the victory in his third start of the season, yielding two runs on five hits in a 5 1/3-inning effort.
Andrew Bailey picked up his 11th save despite surrendering a run in the ninth. He induced Robinson Cano to ground out to end the game with the tying run standing at third base.
Yankees starter A.J. Burnett (8-6) was tagged for three runs on six hits over 5 2/3 frames.
Derek Jeter went 3-for-4 and Nick Swisher cracked a solo homer for New York.