Sheen is telling his friends Richards has taken Sam and Lola's cell phones away so they can't contact their father.
Richards gets $55,000 a month in addition to Sheen having ponied up the $9 million for her home close to his.
Since their agreement was not signed off by a judge, Sheen is hopeful he can get a court to reduce his monthly payouts.
While Sheen and Richards have been mostly friendly in the past few years, some of the acrimony of the years following their 2006 divorce has resurfaced in recent weeks.
On Saturday, Sheen lashed out at Richards on Twitter, smashing a souvenir from their 2002 wedding.
Ray Liotta sues skin care company over use of likeness
Aaron Carter is still in love with Hilary Duff