The Socialist Party was ejected from power in almost all of Spain's regions and lost control in such key cities as Barcelona and Seville in the voting Sunday.
The conservative Popular Party had gained an average of 47 percent of the votes, with the count still to be completed, El Pais reported.
Turnout was above 66 percent, up 3 percentage points from 2007.
Galicia, Andalucía, the Basque country and Catalunya were still being counted.
Zapatero recently announced he will not seek re-election next year.
Upcoming primaries will decide who takes Zapatero's place as the party's candidate for 2012.
The two anticipated contenders are Deputy Prime Minister Alfredo Perez Rubalcaba and Defense Minister Carme Chacon.
The Socialist Party's defeat is thought to be in response to the recession and the poor state of the job market.
Wisconsin business offering 'therapeutic cuddling' forced to close
Megyn Kelly: Santa Claus and Jesus are both white men