With the Rays trailing 3-2 in the bottom of the seventh, Sean Rodriguez drew a one-out walk from Boone Logan and went to second on a wild pitch. David Robertson (0-3) came in to pitch, and Conrad -- pinch-hitting for Will Rhymes -- laced a 3-1 pitch off the wall in the right-field corner for an RBI double.
The next batter, Elliot Johnson, then sent a grounder down the first-base line that got under Teixeira's glove, allowing Conrad to come home with the decisive run.
Matt Moore (5-6) earned the win, allowing three runs on nine hits over seven frames, while Fernando Rodney collected his 23rd save for Tampa Bay.
Teixeira and Robinson Cano each supplied two hits and an RBI for New York.