Arena, the former coach of the U.S. World Cup team, was hired on Monday.
Former Crew players Edson Buddle and Chris Hende combined for a goal and an assist in the 50th minute; and Jerrod Laventure scored the other goal in the 90th minute for the Red Bulls (4-6-8), who flip-flopped with Columbus for last place in the Eastern Conference.
New York rose to a tie with Kansas City and Chicago for third place.
The Crew's club-record winless streak extended to nine games.
Columbus (4-9-6) had five shots on goal to four for the Red Bulls.
New York's Jon Conway made five saves while Noah Palmer made two saves for Columbus, which has 15 points to 44 for front-running D.C. United.
The Red Bulls have 20 points.
Gal Gadot cast as Wonder Woman for 'Batman vs. Superman'
Couple mistakenly served bag of cash at McDonald's drive-thru