The tense duel between American League East rivals will go to the limit, just as baseball's other three division series have done, with the winner being determined in a Game 5 on Friday.
Nate McLouth hit a solo homer for Baltimore in the fifth and the Orioles managed only two base runners through the next seven innings.
Machado, however, doubled to right leading off the 13th against David Phelps and one out later Hardy doubled him home.
That created the latest chapter in the post-season saga of Baltimore closer Jim Johnson, who during the regular season led the majors in saves.
Johnson gave up five runs in the ninth inning of the series opener, saved the second game, blew a save opportunity in the ninth on Wednesday and came on again Thursday in hopes of protecting the one-run lead.
Raul Ibanez pinch hit for Rodriguez on Wednesday and hit a home run off Johnson in the ninth that sent the game into extra innings. New York eventually won that one in the 12th on another Ibanez homer, but this time Johnson won the battle.
The only New York run came in the sixth when Derek Jeter raced in from third on a ground out by Robinson Cano.
The victory went to Pedro Strop, the seventh of eight Baltimore pitchers who baffled the Yankees with two innings of relief before Johnson took the mound.
Swim Week Miami Beach 2014 [PHOTOS]