Citing sources close to the campaign, ABC said Clinton agreed that Penn should step down. Penn will continue to provide polling data to the Clinton campaign.

The campaign has been under pressure to get rid of Penn following news that Colombian officials had hired him to promote a trade agreement that many Democrats in Washington -- including Clinton -- oppose.

Labor unions and Clinton insiders had openly criticized Penn for meeting with Colombian officials.

Penn said in a statement Friday his meeting with the Colombian government was an "error of judgment." Colombian officials called that statement "a lack of respect" and Saturday canceled their $300,000 contract with Penn's public relations firm, Burson-Marsteller, to promote the free-trade agreement with the United States.

Clinton officials maintain Penn's work on the free trade issue was "totally independent" from his campaign duties and insisted the campaign "remains steadfast" in its opposition of the Colombia Free Trade Agreement.