His announcement comes four days after former Federal Reserve chairman Paul Volcker recommended top management step down, so that Volcker could install and head a management team in an effort to save the 89-year-old company.
"This was Mr. Berardino's decision on his own, it has nothing to do with the Volcker recommendation," said Andersen spokesman Patrick Dorton.
"Berardino's decision to resign will bring some sense to this prosecution by the federal government," added Dorton.
Berardino, 51, said he will stay in office until a new CEO is appointed.
In an interview with CNN's "Lou Dobbs Moneyline" program Berardino said, "Frankly, I felt I had to take this step today to put an explanation point behind the voices of our people."
Bernardino, described the Chicago-based firm as being in "deep stress" over its immediate future and said he "can't begin to predict" what the Justice Department will now do.
The key part of Volcker's reorganization plan is the dismissal of a federal indictment against Andersen alleging obstruction of justice in destroying Enron-related documents.
The Justice Department has not commented on a possible dismissal. Volcker hopes to come to an agreement with the government by the end of this week.
He was elected CEO of Andersen in January 2001. Berardino joined the firm in 1972.
"People just don't seem to be listening or taking us real seriously," he told CNN. "We are a serious firm that deserves to continue here in the United States."
In a statement released after Berardino's resignation announcement, Volcker said, "He feels his resignation as chief executive will help clear the air and facilitate the recovery of Arthur Andersen under fresh management."