The United Nations reported the incident Sunday in the waters between Somalia and Yemen had the highest death toll since January 2008 of refugees trying to escape the fighting in Somalia.
"The Gulf of Aden remains one of the deadliest routes for those fleeing the fatal mix of conflict, violence and human rights abuses in the Horn of Africa," said Antonio Guterres, the U.N. High Commissioner for Refugees.
The Yemeni Navy recovered 23 bodies Wednesday of the 57 Somalis aboard, including three smugglers and 54 refugees, six of whom were children, the United Nations said.
Only one man, 42, from Mogadishu, is known to have survived the incident. He swam for almost a day to reach the Yemeni coast, approximately 250 miles east of Aden.
The survivor, who had fled with his wife and three children, said strong waves struck the boat, forcing it to capsize. At that point, only nine men, including the smugglers, were still alive, but the man said he didn't know what happened to them.
As of this incident, 89 people have drowned or are otherwise missing in the Gulf of Aden so far this year, the United Nations reported.
Couple mistakenly served bag of cash at McDonald's drive-thru
Jordana Brewster on Paul Walker: 'He was an enormous presence in my life'