The blaze Tuesday afternoon in Ebute Metta, a district of Lagos where many sawmills are located, was fanned by swirling winds, Nigeria's Channels Television reported.
Paul Apeti, a timber factory owner, said 250 homes had been consumed.
"Millions" of people lived in the area, he said. No casualties were reported.
An estimated 17 million people live in Lagos.
Celebrity Couples of 2014 [PHOTOS]