Emergency responders tried to revive the man after pulling him from the water, but he was pronounced dead after being taken to a New Jersey hospital, The New York Times reported.
John Korff, who helped organize the Ironman United States Championship in New York and New Jersey, told the Times a comprehensive effort was put into water safety for the event, including dozens of lifeguards on land and in the water, as well as police and fire boats on patrol.
"We almost didn't need buoys, we had so many swim-safety personnel," Korff said.
"We are so happy for the city and the athletes, but so saddened by this tragedy."
More than 2,000 competitors were registered for the 140-mile race.
Two people died last summer during the Nautica New York City Triathlon swim and another racer died in a 2008 city triathlon, the newspaper said.