Card will be replaced by Josh Bolten, who currently is director of the Office of Management and Budget, Bush said in a brief televised announcement.
Bush said Card, who will remain as chief of staff until April 14, had been a valued adviser during such times as the Sept. 11, 2001, attacks, the war on terror and White House legislative proposals.
"In all these challenges and crises I have relied on Andy's wise counsel, his calm in crisis and his absolute integrity and tireless dedication to public service," Bush said.
Bolten has also been with the administration a long time, serving in the first term as deputy chief of staff before taking over the budget office.
Bush said: "Josh is a creative policy thinker; he is an expert on the budget and our economy. He is a man of candor and humor and directness. No person is better prepared for this important position."
The Washington Post said Card, on the job for more than five years, has been chief of staff longer than anyone since Sherman Adams in the Eisenhower administration.