"Keep in mind that two-thirds are not in a prison cell right now," said Hilary O. Shelton, senior vice president for advocacy at the National Association for the Advancement of Colored People.
Nearly 6 million American felons have no voting rights, says the Sentencing Project, a non-profit group that works on criminal justice reform issues. Florida leads the nation with the highest rate per capita of disenfranchised felons.
In swing states like Florida and Virginia, another state with a large number of disenfranchised felons, those votes could make the difference in close elections. The deadline to register for the November election is Tuesday in Florida and Oct. 15 in Virginia.
Advocates say they worry the laws are part of larger voter suppression efforts, some designed to keep minorities from casting ballots this fall.
The NAACP launched a national campaign against felon disenfranchisement Tuesday in Tallahassee, Fla. The group is seeking changes in laws that keep felons from voting.
Florida Attorney General Pam Bondi, who sits on the state's executive clemency board, calls the practice fair to law-abiding citizens and victims of crime.
"It is reasonable to ask felons to apply to have their rights restored and to demonstrate rehabilitation by living crime-free during a waiting period after the completion of their sentences," said an official in Bondi's office.
Laws governing the restoration of voting rights vary by state. Most U.S. states restore felons' voting rights automatically after completion of their prison term, parole or probation. Several states allow prisoners with misdemeanor convictions to cast absentee ballots.
But some states with governors have been rolling back voting rights for felons.
Florida, under Republican Gov. Rick Scott, and Virginia, under Republican Gov. Bob McDonnell, are among 12 states -- including Alabama, Arizona, Delaware, Iowa, Kentucky, Mississippi, Nebraska, Nevada, Tennessee and Wyoming -- where felon voting rights may be permanently withheld.
"The problem is the Florida Constitution," said Randy Berg, the executive director of the Florida Justice Institute, a public interest law firm in Miami. He cited a provision added in 1865 that hasn't been repealed.
"Legislators refuse to change the rules on clemency," Berg said.
Scott's administration rescinded a more liberal policy for felons in March 2011. Florida now requires felons to wait 5-7 years before they can apply for restoration of civil rights.
In a statement from Scott's office, ex-felons must demonstrate "willingness to request to have their rights restored." In 2011, 13,000 ex-felons applied for civil rights restoration.
Since Scott's administration amended the law, fewer than 300 ex-felons have voting rights restored. Under the earlier policy introduced in 2007 by Gov. Charlie Crist, who was then a Republican, 155,000 ex-felons had their voting rights restored.
In Iowa, Republican Gov. Terry Branstad rescinded a law in 2011 to automatically restore voter rights, which was instituted in 2005 by former Gov. Tom Vilsack, a Democrat who is now the U.S. secretary of Agriculture.
The danger with executive clemency law is reflected in changes depending on administration.
In Virginia, Shelton said, "If the governor wasn't so moved, (the) people's rights could not be restored."
Thirty-one percent of all voting-age African-American men in Virginia are disenfranchised because of felony records, Shelton said.
Two states -- Vermont and Maine 0-- don't disenfranchise felons. Prisoners registered to vote in Vermont, regardless of where they are incarcerated, may submit absentee ballots.
An official in the Vermont Secretary of State's Office called voting part of the restorative process. Community educators conduct voter registration drives in prisons to ensure that prisoners can participate in elections.
Neither Vermont nor Maine maintain records on how many prisoners register to vote because many use addresses from prior to incarceration.
The NAACP, in cooperation with the Federal Bureau of Prisons, provides former felons with information upon release on how to regain voting rights. Additionally, the organization maintains prison units in Florida, Illinois, Kansas, Mississippi and Missouri for providing absentee ballots.