Ja Rule, real name Jeffery Atkins, left the Mid-State Correctional Facility after serving 21 months Thursday morning and was being held until the Federal Bureau of Prisons determined where he would serve a 28-month sentence for tax evasion, CBS News reported.
The gun charges came after police found a loaded .40-caliber semiautomatic weapon in Atkins' car at a traffic stop in New York City in 2007. Atkins pleaded guilty to criminal possession of a weapon in December 2010 and was sentenced to two years.
He admitted in federal court in March 2011 to failing to pay taxes on more than $3 million between 2004 and 2006.
Atkins may have less than six months left on his sentence because his sentences could be served concurrently, defense attorney Stacey Richman said. He could also be eligible to move to a halfway house.
She added he hopes to get back to making music again as soon as possible, partly motivated by an order to pay $1.1 million in back taxes.
"I in no way attempted to deceive the government or do anything illegal," he told the judge at the time. "I was a young man who made a lot of money - I'm getting a little choked up - I didn't know how to deal with these finances, and I didn't have people to guide me, so I made mistakes."
Celebrity Couples of 2014 [PHOTOS]