McCoy died Saturday, The Columbus Dispatch reported Monday.
As the head of Banc One Corp. and its predecessor City National Bank & Trust, McCoy was an early advocate of the potential of bank mergers, and was known for adopting unorthodox methods to attract customers, the newspaper said.
Confronting angry board members over the use of comedian Phyllis Diller in 1962 radio and TV commercials for his bank, he said: "Gentlemen, it's very simple. You can have either dignity or dividends. I vote for dividends."
McCoy was a major philanthropist in Ohio, supporting Nationwide Children's Hospital and other charitable causes, the Dispatch said.
"John G. McCoy was an innovator and a real giant in our industry," said Jamie Dimon, Chairman and Chief Executive Officer of JPMorgan Chase & Co.
Chase acquired Banc One in 2004, the newspaper said.
"But, more important than that," Dimon said, "he was a great and generous man who provided generations of bankers, including me, with warm counsel and support."
Notable deaths of 2014 [PHOTOS]
2 killed when a tour bus crashes in Delaware