The decision announced Thursday ends a three-year investigation into whether any line officers or their supervisors would be held accountable, The New York Times reported.
Holder said there was insufficient evidence to charge anyone in the death of Gul Rahman, who died in 2002 while shackled to a wall in near-freezing temperatures at a secret CIA prison in Afghanistan, or Manadel al-Jamadi, who died at Abu Ghraib prison in Iraq in 2003 while in CIA custody.
Holder said the decision did not resolve "the propriety" surrounding use of interrogation techniques considered to be torture by many countries.
CIA Director David Petraeus thanked agency employees involved in the inquiry.
"As intelligence officers, our inclination, of course, is to look ahead to the challenges of the future rather than backwards at those of the past," he said.
The chairman of the House Intelligence Committee, Mike Rogers, R-Mich., welcomed the announcement, citing Holder for recognizing that criminal charges in the cases would be "inappropriate."
Elisa Massimino, president of Human Rights First, called the decision "hugely disappointing."
Her organization found the initial investigations of the prisoner deaths had been bungled by military and intelligence officers, she said.