Corey Patterson led off the frame with a single off Yankees starter A.J. Burnett (10-13) and went to second on a sacrifice bunt by Cesar Izturis. Roberts then socked a clean single to right, easily scoring Patterson with the go-ahead run.
Roberts ended the game 3-for-5 with two RBI and Matt Wieters contributed an RBI and walked twice for the Orioles, who took the first of three games at Yankee Stadium.
Brian Matusz (8-12) surrendered three runs on five hits with a pair of walks and four strikeouts over six innings.
Koji Uehara pitched a 1-2-3 ninth to earn his seventh save for the O's.
Burnett yielded all four runs on seven hits and four walks for New York, which entered the game with a 2 1/2-game lead over second-place Tampa Bay in the American League East.
2014: The Year in Music [PHOTOS]