The admission came as two congressional committees, the Secret Service and the Defense Department prepared to investigate alleged involvement with prostitutes by members of the military and Secret Service before last weekend's meeting in Cartagena, Colombia.
Eleven secret service agents and officers were under investigation for allegedly bringing prostitutes to a hotel and drinking heavily.
Col. Scott Malcom, a spokesman for U.S. Southern Command, said the 10 military members "were in the same hotel and when the police were called they somehow got caught up in the incident," the Los Angeles Times reported Monday.
A U.S. military official said the Pentagon investigation is focusing on five Special Forces Army soldiers, two Marines, two Navy personnel and one member of the Air Force. The official said the Navy and Air Force personnel are members of explosive detection unit.
"We let the boss down," Army Gen. Martin Dempsey, chairman of the Joint Chiefs of Staff, said at a Pentagon news conference Monday, referring to President Barack Obama. "I can speak for myself and my fellow chiefs: We're embarrassed by what occurred in Colombia, though we're not sure exactly what it is."
Obama said he would be angry of the allegations proved true.
"If it turns out that some of the allegations that have been made in the press are confirmed, then of course I'll be angry, because my attitude with respect to the Secret Service personnel is no different than what I expect out of my delegation that's sitting here," Obama said in Cartagena at the end of the Summit of the Americas before flying to Washington. "We're representing the people of the United States, and when we travel to another country, I expect us to observe the highest standards because we're not just representing ourselves, we're here on behalf of our people.
"And that means that we conduct ourselves with the utmost dignity and probity. And obviously what's been reported doesn't match up with those standards."
The Secret Service put the agents and officers on administrative leave and immediately replaced them with other service members, the agency said.
U.S. House Oversight and Government Reform Committee Chairman Darrell Issa, R-Calif., told CBS' "Face the Nation" Sunday his committee's investigation wouldn't focus on the agents but on the agency as a whole.
"The question is, is the whole organization in need of some soul searching, some changes, before the president, the vice president, members of the Cabinet are in danger?" he said.
Issa said the Secret Service employees could have made themselves susceptible to blackmail, and said the number of agents involved might have been 20, not just the 11 put on administrative leave. He didn't elaborate on the blackmail allegation or how he came to the higher number of agents allegedly involved.
"Things like this don't happen once if they didn't happen before," he said.
House Homeland Security Committee Chairman Peter King, R-N.Y., whose committee oversees the Secret Service, told The Wall Street Journal his committee, like Issa's, would look at underlying issues.
"How do you take action to make sure it does not happen again and conduct is not being tolerated?" King said.
The agents, who haven't been identified publicly, were caught after a Cartagena prostitute refused to leave a Secret Service agent's hotel room until she was paid, King told the Journal.
She stayed in the room past 7 a.m., in violation of hotel policy, prompting the hotel manager to go to the room, King said, explaining he was briefed on the details by the Secret Service.
When the agent wouldn't open the door, the manager called police, after which the agent let them in.
The woman said she wouldn't leave until she was paid the money she was owed by the agent. The agent denied owing her money but paid anyway, King said.
Cartagena police filed a report to the U.S. Embassy because the situation involved a foreign national, King said. The embassy began an investigation.