He was 92.
Jones, who helped begin a reorganization effort of the nation's military command, died Saturday in Potomac Falls.
Daughter Kathy Franklin said her father died from Parkinson's disease.
He had been living in a military retirement community, The New York Times reported.
Jones served on the Joint Chiefs longer than any one of his predecessors.
He started out as the Air Force chief of staff and then became chairman after serving in the Korean War, the Times said.
In 1982, Jones proposed a sweeping set of changes to enhance the chairman's influence and stop inter-service rivalry.
Although his ideas were controversial, the changes became effective through legislation in 1986.
He also oversaw President Jimmy Carter's failed effort to rescue 53 hostages in Iran.
Jones is survived by two daughters, a son, four grandchildren and two great-grandchildren.
His wife of 67 years, Lois Tarbell Jones, died in 2009.