Klugman, who also starred in the film "12 Angry Men," was at his home in Northridge when he passed, The Hollywood Reporter said.
Klugman's performance on "The Odd Couple" as Oscar Madison -- the sloppy roommate to neurotically fastidious Felix Unger, played by Tony Randall, who died in 2004 -- garnered him two Emmy awards. The iconic show's 114 episodes still are run in syndication today.
His six-decade career also saw stints on Broadway and sporadic appearances in other films, including "Days of Wine and Roses" and "Goodbye, Columbus."
No details were immediately reported on the cause of death. Klugman had parts of his larynx removed in 1989 after being diagnosed with throat cancer.
He leaves behind his wife, two sons and two grandchildren.
Celebrity Couples of 2014 [PHOTOS]