The lead team official said he was "embarrassed" because of the poor season.
The Chiefs were routed 38-3 by the Denver Broncos in Sunday's final regular-season game. While tied with the Jacksonville Jaguars, who fired their general manager Monday, for the worst record, Kansas City will have the No. 1 overall choice in the next NFL draft.
Crennel was hired to coach the Chiefs late in the 2011 season but the team won just four times in the 19 games since then.
Team Chairman Clark Hunt, in a release, said he had "a tremendous amount of respect and admiration for Romeo, both personally and professionally."
"However," Hunt continued. "I am embarrassed by the poor product we gave out fans this season and I believe we have no choice but to move the franchise in a different direction."
Hunt said "the entire football operation" would be reviewed and there hadn't yet been a decision on retaining General Manager Scott Pioli.
The Chiefs defeated the New Orleans Saints in the third week for their first win of the season but lost their next eight games. The day before Kansas City was to play the Carolina Panthers, Chiefs linebacker Jovan Belcher killed his girlfriend then went to Arrowhead Stadium. He asked to talk with Crennel, Pioli and another coach and, with the team officials watching, shot himself.
Crennel's replacement will be the 13th head coach since the franchise was formed in 1960 but third for the team since late in the 2011 season. Todd Haley was fired with three games to go in 2011 and replaced by Crennel who was 4-15 in those three games and one full season as head coach.
Crennel, who coached the Cleveland Browns for four seasons from 2005-08, owns and overall coaching mark of 28-55.
Megyn Kelly: Santa Claus and Jesus are both white men
Wisconsin business offering 'therapeutic cuddling' forced to close