All three victims were found in the fetal position, wrapped in plastic and dumped within 200 yards of each other.
"We are dealing with a sick individual and we have reason to believe that there might be more victims," East Cleveland Mayor Gary Norton told CNN.
Police arrested Madison after it was discovered that he rented the garage were the first victim was found. They obtained a search warrant for his apartment and found "additional evidence of decomposition," Detective Sgt. Scott Gardner said.
Madison reportedly told investigators that he was "influenced" by Anthony Sowell, a death row inmate who murdered 11 women in Cleveland from 2007 to 2009.
"We believe every hour [Madison] was on the street, he would have been a danger to someone," Norton told ABCNews.com.
According to ABC, the incidents occurred only 10 miles from the home where Ariel Castro held three women captive for a decade.
"I have three daughters of my own and me and I do walk and go to the store, so it's kind of scary out here," one resident said of recent events. "I'm about ready to leave. Matter of fact I am moving."