The children will be spending the majority of their time with their father due to the ruling, though it is unclear the extent to which Raymond will get to see the boys, TMZ reported.
Throughout the bitter custody dispute, Raymond had maintained Usher was an absent father, spending too much time absorbed in his own career. She also said he abused prescription drugs in front of the two boys.
Usher had said Raymond was inattentive and irresponsible.
The couple split in 2008 and their divorce was finalized in 2009.
The custody ruling is another blow for Raymond, whose son from a previous marriage died in July after a water-sport accident.
Workers accuse National Zoo of animal mismanagement
Wisconsin business offering 'therapeutic cuddling' forced to close