The National Enquirer reported Kelly's ex-wife Marilyn Dishman posted the news of his Saturday death on Facebook. Cause of death was not immediately reported.
Kelly began his career after graduating from the University of Louisville in Kentucky, the tabloid said.
He won the middleweight title at the 1971 International Karate Championships in Long Beach, Calif., then opened his own martial arts school, or dojo, in Los Angeles.
Acting alongside actors John Saxon and Lee in "Enter the Dragon," Kelly was introduced to big-screen action films.
Kelly became famous after telling the villain in "Enter the Dragon," "Man, you come right out of a comic book," the Enquirer said.
Kelly rode his new-found fame to roles in "blackploitation" films, the newspaper said.
He left behind his movie career in 1983, then became a professional tennis coach.
The Enquirer said Kelly was still mobbed at fan conventions.