Koch, 88, was admitted to NewYork-Presbyterian/Columbia hospital about 10 p.m. Saturday, The New York Times reported.
Koch's former press secretary, George Arzt, said the former mayor could be released as early as Wednesday.
Arzt said Koch didn't look well and complained of swollen ankles at lunch Saturday, and a doctor who dined with him that night recommended he seek medical attention after examining him.
Arzt said Koch was in good spirits when they spoke Sunday evening. "He joked that 'it's not death dealing,'" Arzt said, adding Koch said, "'I could die but not from this.'"
It is the third time he has been hospitalized in recent months, the newspaper said.
Koch was mayor for 12 years beginning in 1978.