Rato did not specify why he was resigning, but some analysts said he may have disagreements with how the government planned to rescue the bank.
In his statement, Rato, the former managing director of the International Monetary Fund and Spain's former finance minister, noted that Bankia closed 800 branches and earned $403 million in profits in 2011.
The bank, however, is holding $41.7 billion in toxic assets, The Wall Street Journal reported Monday.
Bankia was formed in 2010, when the government urged real estate lenders, or cajas, to consolidate. Seven did so to form Bankia.
Spain pumped $5.8 billion into the bank when it was created. At that point, Bankia jettisoned the worst of its frozen assets, which were picked up by BFA, Bankia's parent company, which is unlisted.
Bankia then took itself public -- a rare event for Spain in 2011.
Rajoy's cabinet is expected to assemble a second rescue package for the bank this week. It is expected Bankia will be infused with $9 billion to $13 billion, the Journal said.