Two years ago, the League of United Latin American Citizens and La Raza, the nation's largest Hispanic rights group, enthusiastically supported U.S. President George Bush's nomination of Gonzales but representatives from both groups recently told the Post it was a mistake.
"I have to say we were in error when we supported him to begin with," said Brent Wilkes, executive director of LULAC. "We hoped for better. Instead it looks like he's done the bidding of the White House."
His counterpart at La Raza, Janet Murguia, agreed.
"I think we've been disappointed with his record so far," she said.
Gonzales has come under fire for the dismissal of eight U.S. attorneys who allegedly weren't toeing the White House line.
Some Democrats in Congress are calling for his resignation but not all Hispanic groups have given up on him, the newspaper said.
"We strongly oppose what is nothing but patently political calls for the resignation of Alberto Gonzales," the Latino Coalition wrote in a statement.
Celebrity Couples of 2014 [PHOTOS]