Daniel Cabrera (1-0) threw six-plus innings to get the win, giving up two runs on five hits.
Baltimore has won two in a row and the Yankees have dropped two straight.
Phil Hughes (0-3) surrendered five runs on eight hits in 5 1/3 for New York.
The game was tied prior to Baltimore producing its big inning.
Payton came up as a pinch-hitter and produced a two-run single. Mora then did the same and Markakis drove in the final run of the inning with a base hit.