Mbeki won less than 40 percent of the congressional vote despite rape and corruption allegations against Zuma, The New York Times reported Wednesday.
Zuma's victory at the African National Congress meeting paves the way to his ascendancy to the South African presidency when Mbeki's term expires in early 2009.
A successful prosecution of either the rape or bribery charge means Zuma could not run for president under South Africa's constitution.
Zuma's supporters see him as a champion of the poor amid 25 percent unemployment rates and a widening wealth gap.
Mbeki led South Africa's government and economy with remarkable efficiency, but his critics say he largely stymied lower party officials from participating in government decisions.