Gerald Williams "Butch" Perry Jr., 60 of Guthrie, Okla. faces charges of first-degree rape, forcible oral sodomy and rape by instrumentation, The Oklahoman reported Tuesday.
Officials said Perry worked at the Westview Living Center where reports of rape were brought to the staff's attention May 6.
One of the patients told a staff member that Perry had touched her inappropriately and that he had sex with her in her room.
Four other women came forward during the police investigation and said Perry had touched them or received oral sex from them.
An affidavit filed with the court said all of the victims have been adjudicated mentally deficient and four of them function at middle school level.
Perry is being held in the Logan County jail with bail set at $250,000.
Puzzle-maker slips 'Murdoch Is Evil' into Rupert Murdoch's Sunday Telegraph
Exploding whale video goes viral on Internet