U.S. Ambassador to Libya Chris Stevens and three of his staff members died in what the U.S. State Department said was a terrorist attack Sept. 11, 2012, on the diplomatic post in Benghazi.
Panetta, speaking Thursday at a U.S. Senate Armed Services Committee hearing, said there was no intelligence that would have given the Defense or State departments an indication that an attack on the Benghazi compound was imminent.
"Frankly without an adequate warning, there was not enough time given the speed of the attack for armed military assets to respond," he said.
The attack was initially tied to regional unrest surrounding a controversial film produced in the United States viewed as denigrating to the Islamic Prophet Muhammad but that explanation has been discounted the Obama administration has since termed the event a terrorist attack.
Panetta said the incident wasn't of a nature that U.S. military force could have prevented in time.
"Time, distance, the lack of an adequate warning, events that moved very quickly on the ground prevented a more immediate response," he said.