His firing had been rumored for several months and especially after Donnie Walsh became the Knicks' president April 2.
In two seasons, Thomas, a hall of fame player with Detroit, had a 56-108 record.
"Following a lot of discussion and thought, I made the decision that Isiah will no longer coach the team," Walsh said in a statement reported by New York Newsday.
The paper added that Thomas will remain with the team but will not have an official title and will report directly to Walsh.
The Knicks made the playoffs one time while Thomas was in charge of basketball operations.
They were swept by New Jersey in 2004 and his tenure also was marred by a sexual harassment lawsuit filed by former team executive Anucha Browne Sanders.
She received $11.5 million last November in an out-of-court settlement.