Duffy, who was part of the "Black Trinity" photographers along with David Bailey and Terence Donovan, was best known for his iconic fashion photos that appeared in such glossies as Glamour, Harper's and Esquire. He also created album art for three of David Bowie's records, the most famous of which is the "Aladdin Sane" photo.
Duffy notoriously destroyed some of his photos in 1979 after one of his staff members asked him where he kept his toilet paper.
In a 2009 interview, Duffy said of the incident, "I realized I was chairman, CEO and senior stockholder in my business, and now I was responsible for the toilet paper," The Daily Telegraph reported.
Duffy, who died Monday, is survived by his wife, June, and four children.