The blaze Tuesday afternoon in Ebute Metta, a district of Lagos where many sawmills are located, was fanned by swirling winds, Nigeria's Channels Television reported.
Paul Apeti, a timber factory owner, said 250 homes had been consumed.
"Millions" of people lived in the area, he said. No casualties were reported.
An estimated 17 million people live in Lagos.
NBC reportedly holds celebs hostage to Jimmy Fallon's show
Ray Liotta sues skin care company over use of likeness