Annan, speaking with CNN, said Movement for Democratic Change leader Morgan Tsvangirai must use the actions of Mugabe's Zanu-PF party to learn what to avoid if Tsvangirai wants to lead Zimbabwe.
"There is concern that whatever the outcome of the election, there is need for dialogue, there is need for mediation between the two groups regardless of who wins," Annan said.
Annan, the former U.N. secretary-general, has been involved in arbitration after the March 29 presidential election stalemate rocked the country. Tsvangirai finished ahead of Mugabe, who ruled Zimbabwe for 28 years, but didn't win a majority, necessitating a runoff.
Annan said a win for Tsvangirai in the June 27 runoff must not be allowed to become an anti-Mugabe attack.
"The opposition has not had a chance to govern because alternates that you expect in democracy have not happened and so it is extremely difficult," Annan told CNN. "(If) the opposition wins, they will have to find a way to live with Zanu-PF who have controlled the levers of power for so long."