On a 5-1 vote Wednesday, the county commission denied Mike Casey a permit that would have allowed him to keep the five primates in his rental home in the county.
Casey has a month to make other living arrangements for the animals, the Las Vegas Sun reported. He said he will not give them up, calling them "his family."
Commissioner Tom Collins was the only one to vote in favor of letting Casey keep the chimps and monkey at the Las Vegas-area home.
"People should be able to walk their dogs, children [should be able to] play and enjoy their yards without worrying that a chimp should get loose," one woman told the commission.
The People for the Ethical Treatment of Animals also weighed in against Casey but one woman said the home had previously housed two tigers and a cougar.
"We never had any issues regarding these animals," she said.
One man noted dogs have proven to be more deadly than chimpanzees in the United States.
"Are chimps dangerous? Sure they are. So are dogs and cats. So are cars and skydiving ... ." he said.
The newspaper did not report the names of those testifying at the commission hearing.
Celebrity Families of 2014 [PHOTOS]