Perry, who would have turned 90 this week, was found dead Sunday, apparently of natural causes, The State newspaper in Columbia reported Monday.
His body was found by a family member who each Sunday went to the judge's home to prepare a meal for Perry and his wife Hallie, who is in poor health, said Richland County Coroner Gary Watts.
Perry was at work Friday at the courthouse that bears his name and he regularly worked a full schedule.
"He was a shining example of unflinching courage and leadership," said Columbia Mayor Steve Benjamin. "Simply put, he was a giant and this world will be a lesser place without him."
Flags in Columbia will be flown at half-staff in the coming days, Benjamin said.
"Matthew Perry -- an iron fist in a velvet glove -- courteous, polite, even jocular … but unshakably determined," said South Carolina historian Walter Edgar.
Perry was one of the first black men from the South appointed to a federal court. He was still serving as a senior U.S. District Court judge for South Carolina at the time of his death.
"I would like to see an improved quality of life for citizens around the state," Perry said last week. "I am very much concerned about those who remain uneducated and who are impoverished, living at the edge of society. "
Information about arrangements wasn't immediately available.