James reportedly entered the facility intending to spend 45 days dealing "with personal issues" in the wake of rampant reports he had sex with multiple other women during his 5-year marriage to actress Sandra Bullock.
Although his departure from the rehab came amid rumors claiming Bullock has no intention of taking him back, E! News said James left the clinic due to security leaks.
"He was fed up with all the media leaks from inside. He had privacy and security concerns," a source told the celebrity news Web site.
Bullock has not spoken publicly about the scandal since it broke last month, shortly after she won her Oscar for "The Blind Side."