The victims were identified as the first-term councilman and Othniel Askew, both Brooklyn residents.
The 2:08 p.m. EDT incident occurred inside the City Council chamber while the council was in session. Both victims were found lying on a balcony overlooking the chamber.
"I don't think I've ever had as tough a day in 61 years as today," Mayor Michael Bloomberg who told reporters. He said Askew was seen standing over Davis' body on the balcony, firing several shots at Davis, who was laying on the balcony's floor. Askew was then shot and killed by a police officer assigned to City Hall.
Officials said Askew was wounded in his chest and arms while Davis was fatally wounded about seven times in his torso with a semi-automatic weapon.
Ammunition was later found in Askew's socks. It was not believed Davis, a former police officer, had time to draw or fire the weapon he was carrying.
Bloomberg said Davis entered city hall along with Askew, who had filed papers to run against Davis in the primary election this fall.
Bloomberg said: "We have films of them walking in together, talking, and going through a security booth but (they) did not go through the metal detectors. It was clear Mr. Askew came in as a guest of James Davis and, therefore, did not have to go through the (metal detector).
"Effective immediately I've ordered that everyone -- the mayor, speaker and other officials -- will have to go through the magnetometers."
Councilman Michael Nelson said council was conducting a routine session when suddenly shots were fired on the balcony and people started screaming. Nelson said many council members dropped to the floor for safety.
Immediately after the shootings, officials implemented an emergency plan, ordering the building evacuated, local subway traffic halted and temporarily closed the nearby Brooklyn and Manhattan bridges. Officials also closed numerous city streets, creating massive traffic backups.