Police said Monday that all four men were arrested for "suspected acts of terrorism."
The suspects, all between 40 and 50 years old, were taken into custody Sunday in three different provinces, The Wall Street Journal reported.
Keith Khoza, an ANC spokesman, said the party believed the men planned to plant a bomb at the party's convention, during which it will choose its leaders and adopt policies.
"We are concerned that people have these intentions," Khoza said at the conference. "It doesn't augur well for a developing democracy and at a time when we want national unity. This only highlights the tensions."
Police spokesman Brig. Billy Jones said he could not link the alleged plot to the party's conference.
"At this stage we cannot confirm any connection to the ANC event," he said.
The Federal Freedom Party, a right-wing party that believes Afrikaans, or the white minority in South Africa, should govern their own state, confirmed that two of the men arrested were members of the party.