Emergency workers using concrete cutters and cranes to try to reach survivors, CNN reported. Workers said 300 to 400 people might be trapped in the building near Dhaka.
At least 600 people were being treated at hospitals, CNN said. A doctor said many of the injured had lost arms or legs.
"Many more people, dead or alive, are still trapped inside," Brig. Gen. Ali Ahmed Khan, head of the fire service, told reporters, the national news agency, Bangladesh Sangbad Sangstha, said.
The building, which housed an indoor market, clothing factories and a bank, had developed cracks the previous day and collapsed Wednesday morning, The Daily Star newspaper in Bangladesh reported.
The Institute for Global Labour and Human Rights said workers were told to return to their jobs after they noticed a crack in the building and an inspector determined the structure was unsafe.
"Our staff are on the ground," Charles Kernaghan, an institute spokesman, said. "The garment workers saw the crack yesterday and refused to work. A Savar sub-district officer came and told the owners that the building was unsafe. This morning the factory owners told the workers that 'some cracks will not be a problem.' They ordered the workers back to work. The factory collapsed an hour later."
There was no immediate estimate of how many people were in the building at the time of the accident in the Savar neighborhood of Dhaka.
Opposition parties staging a strike against the government of Prime Minister Sheikh Hasina suspended the action in the Savar area to help in the rescue effort, the report said.
"I have not seen so many patients being brought in like this before," a doctor told the BBC.
The Daily Star said only the ground floor of the building remained intact.
A similar building collapse in Bangladesh in April 2005 killed 64 workers. Last November, fire engulfed a clothing factory in Dhaka, killing at least 110 people.