Haneke, who wrote and directed the film, which took inspiration from events in his family, was named Best Director and Emmanuelle Riva earned Best Actress for her role in the French-language drama that tells the story of two elderly, retired music teachers whose daughter lives abroad.
The movie also stars Jean-Louis Trintignant and Isabelle Huppert.
The award for Best Non-Fiction film went to "The Gatekeepers," while Best Cinematography was awarded to "Master" and "This is Not a Film" won in the Experimental category.
Laurence Kardish, senior film curator at New York's Museum of Modern Art, was given the Heritage award for his 44 years of service to the industry and to Milestone Film and Video for its ongoing Shirley Clarke project.
The society dedicated this year's awards to the late Andrew Sarris, which it described as "one of the most original and influential American film critics as well as a founding member of the society."
2014: The Year in Fashion [PHOTOS]
'Vacation' re-boot starts shooting in Georgia