Kelly's former wife Marilyn Dishman told CNN the former martial arts instructor died of cancer at his home in San Diego, Calif. but did not elaborate on the details.
Kelly is best known for his performance in the 1973 film "Enter the Dragon," where he starred alongside Bruce Lee.
After a brief career in action movies, Kelly became a professional tennis player. He was a ranked player on the USTA senior circuit.
"I never left the movie business," Kelly told the Los Angeles Times in 2010. "It's just that after a certain point, I didn't get the type of projects that I wanted to do."
Kelly is survived by his partner of 33 years and his ex-wife.