The Orioles had trailed 1-0 since the second inning as Yankees starter Hiroki Kuroda stifled them on three hits over seven shutout frames.
Rivera (1-2) blew only his second save of the season, breaking a streak of 41 straight converted attempts at Yankee Stadium.
Darren O'Day (5-0) was credited with the victory thanks to a scoreless eighth inning and Jim Johnson racked up his major league-best 30th save as the Orioles salvaged one game of a weekend series and snapped New York's six-game winning streak.
Edwin Nunez drove in the Yankees' only run.
Wisconsin business offering 'therapeutic cuddling' forced to close
'SNL': 'Anchorman 2' cast, One Direction sing 'Afternoon Delight' [VIDEO]