"The New York that Ed inherited is almost unimaginable today," Mayor Michael Bloomberg said during the service at Temple Emanu-El, noting the city was plagued by graffiti, crime, bad government and whole neighborhoods that "looked like they had been bombed out in an air raid."
"He restored the arc of our city's history," Bloomberg said. "He reminded us why we loved New York, and he inspired us to fight for it."
Koch's spokesman, George Arzt, said the former mayor, who also served in Congress from 1969 to 1977, died of congestive heart failure at New York-Presbyterian Columbia Hospital, where was being treated. Koch had coronary and other medical problems since he left office in 1989, but had always been in relatively good health in his on-the-go life as a television judge, radio talk-show host, author, lawyer, newspaper columnist and professor, among other pursuits.
Former President Bill Clinton returned early from a trip to Japan to speak at Koch's funeral, The New York Times reported.
"I come here to speak for myself and also for Hillary, who loved him very much and was grateful for his endorsement in every race she ran," Clinton said. "We miss you so much because we all know we are doing a lot better because you lived and served."
After the service, Koch's coffin was taken out of the synagogue as "New York, New York" played on the organ and the audience erupting into spontaneous applause as the coffin passed.
Hundreds of mourners lined the streets in the cold before the funeral, the Times said. Politicians of all stripe, many of whom had clashed with Koch, a Democrat when he was mayor, sat side by side inside the temple.
Koch's coffin was draped in the New York City flag as it was carried into the temple and six uniformed officers stood alongside the coffin as thousands filled the seats.
The honor guard represented the city's police and fire departments, the parks, sanitation and correction departments, and the sheriff's office, the Times said.
Former Mayor Rudy Giuliani said Koch would be remembered for what he achieved in office and the optimism he brought to the job.
"It's a sad day," Giuliani said. "Not too many people, if anyone, contributed more to New York City than Ed Koch. It's a life full of contributions and achievement. He was so optimistic. We should feel guilty feeling too sad. He wouldn't want us to."