The bank conducted a yearlong search for Pandit's replacement, The New York Times reported. Pandit had said he would step down when the bank was on solid ground, and the bank announced Monday it had earned $3.27 billion in the third quarter.
Still, his departure is something of a surprise, given there was no immediate warning that Pandit would leave Tuesday with his departure "effective immediately."
Pandit will be succeeded by Michael Corbat, the bank's head of European and Middle Eastern businesses.
Pandit led the bank through the financial crisis, which was just beginning to build up steam when he took over in December 2007.
Before he took the top job, the bank announced $18 billion in losses tied to collateralized debt obligations, which are securities used as insurance for the mortgage industry. Citigroup accepted $45 billion from the government to keep itself safely afloat during the financial crisis.
At the time, Pandit said he would not accept more than $1 in annual salary until the bank became profitable again. That came around by the end of 2010, when the bank repaid the government loan, including a $12 billion profit for taxpayers.
2014: NFL Cheerleaders [PHOTOS]
GM recalls 221,000 Cadillacs and Impalas