Speaking Tuesday on CBS' "This Morning," Panetta said public disclosure of "the particulars of how those operations are conducted" gives enemies information on how the United States operates.
"I think when someone who signs an obligation that he will not reveal the secrets of -- of this kind of operation and then does that and doesn't abide by the rules, that when he reveals that kind of information it does indeed jeopardize other operations and the lives of others that are involved in those operations," he said.
Panetta said the author of the book, former Navy SEAL Matt Bissonnette, should be concerned about his safety.
"He was very much a part of the operation that got bin Laden," Panetta said. "You know, there's no question that -- that should make him concerned. Makes us concerned about his -- his safety."
The defense secretary said the American people have a right to know about the May 2011 operation that resulted in the death of the al-Qaida leader in Pakistan, but he said the Department of Defense is "currently reviewing" the question of whether Bissonnette crossed a line by writing "No Easy Day."
The Pentagon had said the book contains sensitive information and was a "material breach" of non-disclosure agreements Bissonnette signed.
Panetta also said he is "very concerned" about defense spending cuts set to take effect in 2013 as part of a deal on Capitol Hill to avoid a default by the federal government.
He said allowing the cuts to take place will "hollow out our force and it's going to weaken our defense system" but he acknowledged the Pentagon has not begun preparing for the cuts. Panetta said "the whole purpose" of the legislation was to force congressional leaders to "do the right thing and prevent this from happening."
"Well, now they've cocked the gun," he said.
Notable deaths of 2014 [PHOTOS]