The blaze Tuesday afternoon in Ebute Metta, a district of Lagos where many sawmills are located, was fanned by swirling winds, Nigeria's Channels Television reported.
Paul Apeti, a timber factory owner, said 250 homes had been consumed.
"Millions" of people lived in the area, he said. No casualties were reported.
An estimated 17 million people live in Lagos.
Jordana Brewster on Paul Walker: 'He was an enormous presence in my life'
Kate Middleton recycles dress at movie premiere