Police described the suspect as being in his early 30s, with a slight build, 5 feet 6 inches tall and having a light complexion. The man apparently wears different colored woolen hats and a red ring on his left hand. He has a cut on his lower lip and is believed to be either from KwaZulu-Natal or Swaziland.
Police offered up a $10,168 reward for information leading to arrest and prosecution of the man, the South African Government News Agency reported.
The slaying of the two children -- cousins aged 2 and 3 -- is one in a spate of child killings in South Africa recently, the news agency said. Two other children were brutally assaulted and killed in Kathlehong.
South African President Jacob Zuma said Wednesday he directed law enforcement agencies to do everything in their power to find the suspects in the recent killings.
"We urge the communities to contain their anger and exercise restraint. Such matters must be left to the police to handle. They will find the culprits," he said. "This reminds us that we need to re-inculcate the time-honored value that every child is my child ... we must work together to protect our children."