Police discovered Patrick Karegeya's body Wednesday in his room at the Michelangelo, a luxury hotel, as well as a bloody towel and a rope in the room's safe, the Wall Street Journal reported.
Karegeya, who checked into the hotel Sunday, was found with his neck swollen.
Police said in a statement that "investigations are under way" but did not indicate whether there were any suspects in the case.
Karegeya ran Rwanda's foreign intelligence service for a decade but left the country in 2007 after he had a falling out with President Paul Kagame.
Kagame accused Karegeya of being behind a series of grenade attacks in Kigali, which Karegeya denied. In turn, Karegeya accused Kagame of ignoring human rights, inhibiting freedom of speech and muzzling political opposition.
In 2010, Karegeya founded an opposition party called the Rwanda National Congress together with Faustin Kayumba Nyamwasa, a former Rwandan army chief of staff who also lives in South Africa.
In 2011, Rwanda issued arrest warrants for Karegeya, Nyamwasa and other dissidents. A military court convicted them in absentia of threatening state security and promoting ethnic divisions.
Ohio bar shooting arrested, charged with murder
McPhee, Cokas 'working on their marriage' after affair