There was one ugly incident, when a small group attacked a mounted police officer, The Arizona Republic reported. Five people were arrested.
The marchers walked 4 miles from Falcon Park to Tent City, a detention facility set up by Arpaio. Singer Linda Ronstadt walked with the protesters, who ranged from local teachers and activists to a group of meat packers who drove 12 hours from San Francisco.
Arpaio, who has won five elections in Maricopa County, has long been a lightning rod, known for tactics like making inmates wear pink underwear. In recent years, he has aggressively enforced immigration laws under an agreement with the Department of Homeland Security that allows deputies to check the immigration status of everyone they arrest.
Local Hispanics accuse Arpaio of using racial profiling.
Homeland Security Secretary Janet Napolitano, a former governor of Arizona, was also a target of the protest. Marchers urged her to be more lenient with illegal immigrants.
"Babies are being separated from their mamas, and families are being split up in the process, all in the name of anti-immigration," said the Rev. Liana Rowe of Shadow Rock United Church of Christ in Phoenix.