Howard St. John Hunt, 52, of Eureka, Calif., who goes by St. John, is writing an account of the story, while his brother David, 43, of West Los Angeles, is looking for a publisher, the Los Angeles Times said Tuesday.
St. John said his father, who died in January, wanted to include the story in his memoir, "American Spy: My Secret History in the CIA, Watergate and Beyond," which came out this month, but cut it in fear of perjury charges.
The elder Hunt was suspected of sharing a hatred of Kennedy with Cuban exiles angered that the president didn't order air cover to rescue the 1961 Bay of Pigs invasion that Hunt helped organize.
"He told me in no uncertain terms about a plot originating in Miami, to take place in Miami," said St. John Hunt.
No other Hunt family members corroborate the veracity of the story, and the newspaper said the brothers' materials were inconclusive.
Hunt's attorney, Bill Snyder, said Hunt "was just speculating. He had no hard evidence."
Swim Week Miami Beach 2014 [PHOTOS]