More than 400 children and their families have been transferred out of the Auburn Family Residence in Brooklyn and the Catherine Street shelter in Lower Manhattan, the New York Times reported.
The administration will be converting the two facilities into adult family shelters and will be finding other housing for the displaced families, officials said.
Inspectors have reported that the two shelters have cockroaches, insufficient heat, spoiled food and other problems, including violence.
"Until today, no mayor was willing to say no children should be treated this way, and that's a historic breakthrough," said attorney Steven Banks at the Legal Aid Society, which has fought the city over shelter conditions.
Auburn has been cited for more than 400 violations.
Catherine Street has had at least 150 violations since 2006.
The city has set a hearing on family shelters for next week.
The transition plan, which includes enhanced security measures and upgrades to both facilities, will cost the city more than $13 million.