The Los Angeles Times quoted Robertson -- appearing Sunday on ABC's "This Week with George Stephanopoulos" - as saying he does not believe Muslims are suited for such positions.
"They have said in the Koran there's a war against all the infidels," the newspapers quoted Robertson as saying. "Do you want somebody like that sitting as a judge? I wouldn't."
The Council on American-Islamic Relations called that statement "hate-filled rhetoric," and urged others to disassociate themselves from Robertson's remarks.
"This type of hate-filled rhetoric deserves repudiation from all who respect America's long-standing tradition of pluralism," said Rabiah Ahmed, CAIR's communication coordinator. Ahmed said many Muslims already serve with distinction in many levels of government, including judgeships at the state and local level.
CAIR said Robertson had previously made similar comments concerning Islam and Muslims.
Ray Liotta sues skin care company over use of likeness
Aaron Carter is still in love with Hilary Duff