Patricia Kirtley told the New York Daily News she raised their four daughters and even had her sisters take in children he had with another woman.
"We survived," Kirtley, who is partially blind, said Thursday in Reynoldsburg, Ohio. "My children are survivors. They know if we get a little bit that God provides we make it into a lot."
Kirtley had to go on welfare and eventually went to school and got licensed as a blind vendor.
"My mother and sisters pitched in and drove me because I can't see to drive," said Kirtley, now 58. She now has 16 grandchildren.
Kirtley said she stayed on friendly terms with Williams but he rarely visited.
Williams has had several job offers as a radio announcer since The Columbus Dispatch posted a video of him panhandling and touting his "golden voice" this week.
Notable deaths of 2014 [PHOTOS]