His visit to parliament came on the anniversary of his walk to freedom two decades ago after 27 years in prison, CNN reported. Earlier reports indicated Mandela wouldn't attend.
"President Mandela united this country behind the goal of a non-sexist, non-racial democratic and prosperous South Africa," President Jacob Zuma said. "We should reflect on how far we have traveled as a country."
Mandela, 91, and his wife, Graca Machel, were present to hear Zuma speak, as was Mandela's ex-wife, Winnie Mandela.
Nelson Mandela was imprisoned for 27 years for opposing South Africa's government, which brutally enforced its strict segregation laws. After he was released, Mandela worked with South Africa's white president, F.W. de Klerk, to end apartheid.
The African National Congress elected Mandela as its president, and he was elected South Africa's first black president by a landslide in 1994.
Many of South Africa's political leaders marched past the gates of Drakenstein Correctional Center, known as Victor Verster prison when Mandela was jailed there, re-enacting Nelson Mandela's walk to freedom 20 years ago.