Hagel said that while "someone has to be accountable" for the allegations of mismanagement of patient wait times at VA health clinics, firing Shinseki would do little to "fix the problem."
But he said there should be "zero tolerance" once the inspector general at the VA finishes its investigation.
"Why did this happen? How was it allowed to happen? Who's accountable? Someone has to be accountable here, like in any institution," Hagel told Charlie Rose.
Shinseki spoke directly to veterans Thursday, reiterating his commitment to "re-doubling its efforts, with integrity and compassion, to earn your trust."
"I take any allegations about patient safety or employee misconduct very seriously," he said. "The reports of Veterans' negative experiences while seeking VA care are of great personal concern to me."
2014: The Year in Music [PHOTOS]