Bad weather Thursday morning made the rescue operation difficult, the BBC reported.
"Search operations continue but it is now almost impossible survivors will be found," Zanzibar police spokesman Mohamed Mhina told reporters. "The weather was very bad, there were big waves and strong wind."
Zanzibar Transit Minister Hamad Masoud Hamad said the vessel, carrying about 290 people, capsized near the island Wednesday, the BBC said.
Officials said about 31 of the passengers were children, CNN reported.
About 145 people have been rescued but dozens remained missing, the minister said.
Operations were suspended at nightfall Wednesday and resumed Thursday, with army helicopters, government troops and boats, CNN said.
The boat was traveling from the Tanzanian mainland city of Dar es Salaam to Zanzibar but encountered difficulties because of strong winds, naval officials said. Zanzibar, a semi-autonomous part of Tanzania, is a popular tourist destination because of its beaches.