Mark Teixeira led off the 10th with a walk off Michael Gonzalez (0-1), went to third on Alex Rodriguez's double and came home on Swisher's line-drive fly ball to right, capping the Yanks' comeback from a 5-0 deficit.
New York tied the game when Jorge Posada socked a solo homer in the bottom of the ninth off Orioles closer Kevin Gregg.
Rodriguez went 3-for-3 with an RBI and Robinson Cano added two hits and a run driven in for New York, which improved to 7-4 on the season.
Mariano Rivera (1-0) was credited with the victory after hurling a scoreless top of the 10th.
Nick Markakis singled, doubled and hit a two-run home run, driving in three runs, and Cesar Izturis added two hits and two runs scored for the O's.