Rodriguez moved into sole possession of fourth place on baseball's all- time home run list with a solo shot to center in the third inning. His 661st career blast broke a tie with the great Willie Mays and leaves him 101 shy of equally-embattled home run king Barry Bonds.
Rodriguez and Mark Teixeira both knocked in a pair of runs for the division- leading Yankees, who were coming off a successful 4-2 road trip.
Nathan Eovaldi (3-0) pitched into the sixth and earned the win despite a shaky outing. He allowed three runs while walking three and giving up six hits, including solo home runs to Jimmy Paredes and Caleb Joseph.
The Orioles dropped their third straight after Chris Tillman (2-4) was tagged for four runs on 10 hits over 5 2/3 innings.
Paredes golfed a payoff pitch over the bullpen beyond the wall in right-center in the first inning, and the Yankees pushed a pair across in the bottom half.
Rodriguez nearly passed Mays when he sent a deep fly ball to right field with runners on the corners, but Delmon Young made a leaping catch at the wall to rob him of a homer. Jacoby Ellsbury scored on the long sacrifice fly, and Teixeira hit an RBI single that ended in him being thrown out by Young trying to stretch it into a double.
Joseph and Rodriguez traded homers in the third, and the Yankee Stadium crowd gave the embattled star a standing ovation following his historic home run.
Consecutive stand-up doubles by Travis Snider and Joseph tied the game again in the fifth, but Eovaldi limited the damage by picking off Paredes at first base with runners on the corners and one out.
Teixeira's RBI double into the right-field corner broke a 3-3 tie in the fifth, and the bullpen bailed both starters out in the sixth by stranding two runners.
New York's relievers retired all 10 of the 11 hitters they faced. Justin Wilson recorded four consecutive outs, Dellin Betances pitched a perfect eighth, and Andrew Miller worked around a leadoff walk in the ninth to post his 12th save in as many chances.
J.J. Hardy was activated off the disabled list and went 0-for-4 in his season debut. The Orioles shortstop missed the first 25 games of the year with a left shoulder strain ... Ellsbury extended his hitting streak to 10 games ... New York's 1-through-4 hitters -- Ellsbury, Brett Gardner, Rodriguez and Teixeira -- combined to go 8-for-13 with four runs scored, four RBI and two walks.