Citizens for a Better Arizona President Randy Parraz and Maricopa County Supervisor Mary Rose Wilcox asked Arpaio to resign Tuesday at an event hosted by Parraz's group, The Arizona Republic reported.
Other groups, including Valley Black and Asian Chamber of Commerce leaders, joined the call for the sheriff's ouster.
"We are joining with our African-American and Hispanic-American brothers and sisters to ask Sheriff Joe Arpaio to resign," said Asian Chamber of Commerce Director Madeline Ong-Sakata.
"His obsession with Hispanics, legal or illegal," makes Arpaio "ineffective," she said.
Asian Pacific Community in Action President Doug Hirano said members of his organization considered "racial profiling by the Sheriff's Office reminiscent of policies of countries Asian-Americans have fled."
"The majority support me in this fight against illegal immigration," said Arpaio, who plans to run for a sixth term as sheriff.
The U.S. Justice Department accused Arpaio's office in December of unconstitutional racial profiling in traffic stops and arrests of Latinos. The department said a 3 1/2-year investigation found the office discriminated against Latinos with limited English-speaking ability in jails, denied them services other inmates receive and retaliated against critics.