A statement released by family members to CNN confirmed that Frazier, who fought Muhammad Ali in three classic 1970s bouts.
Frazier beat Ali once in the three meetings -- 1971's "Fight of the Century" at Madison Square Garden in New York -- landing a devastating left hook on Ali's jaw and handing him his only professional defeat.
His most famous bout, however, was 1975's "Thrilla in Manila" against Ali, in which Frazier's trainer stopped the fight in the 14th round.
Frazier lost to George Foreman in a five-round technical knockout in his following fight, held the next June.
A member of the boxing Hall of Fame, he finished with a 32-4-1 professional record with 27 knockouts.
Frazier took home the gold medal from the 1964 Olympics in Tokyo.