He replaces Charles Prince, who resigned Nov. 4 as Citigroup disclosed it faced up to $11 billion in fourth-quarter losses tied to the bank's exposure to mortgage-related investments.
The losses were on top of some $6.4 billion in third-quarter write-downs, including about $1.6 billion stemming from subprime mortgages.
Rubin will return to his previous duties as board member and chairman of the board's executive committee, the bank said.
Pandit -- a former Morgan Stanley executive who joined Citigroup this year when the bank bought his Old Lane Partners hedge fund for $800 million -- said he would work with Bischoff, Rubin and Citigroup's board "to assure that our strategy, structure, scale and diversification position the company for growth."
"Simplifying the company's organizational structure and aligning our businesses and resources with appropriate goals and economic realities will be among our initial priorities," he said.
Citigroup's stock price, which had plunged 38 percent this year, closed down $1.54, or 4.43 percent, at $33.23 on the New York Stock Exchange.