The military intelligence department, known as J2, is accused of subjecting prisoners to beatings, electric shock and sensory deprivation to force confessions. Amnesty International said it has documented 45 instances of unlawful detention and 18 cases of torture at a prison in Kigali. Some of the men were detained for as long as nine months without access to a lawyer.
Two men have gone missing entirely since March 2010, the group said.
"The Rwandan military's human rights record abroad is increasingly scrutinized, but their unlawful detention and torture of civilians in Rwanda is shrouded in secrecy," said Sarah Jackson, Amnesty International's acting deputy Africa director.