A baby and two first responders were among the victims after a swollen river flooded the center of the city, the BBC reported.
More "drastic measures" may be necessary to evacuate people in high-risk areas who ignored warnings, President Dilma Rousseff said.
"Our prevention system warns the people," Rousseff told Brazilian reporters in Rome after meeting with Food and Agriculture Organization head Jose Graziano.
"What I think is that a little more drastic measures will have to be taken so people don't stay where they are not supposed to be."
Some areas received nearly 12 inches of rain within 24 hours, and at least 50 people lost their homes to the landslides, authorities said. More heavy rain was in the forecast.
Similar slides in the area in 2011 killed 900 people.