The officials said the Middle East peace process could be at risk, CNN reported.
"We think it is not useful for people to be running to Hamas at this point and having meetings with Hamas," U.S. national security adviser Stephen Hadley said.
Most Israeli officials refused to meet with Carter because of his meetings with Hamas.
Carter's visit included a Friday session with Hamas' top politician, Khalid Mashaal.
Some diplomats have a view similar to Carter's.
"You should never give in to the terrorists ... but you should never be the ones refusing to talk," said Jonathan Powell, who was involved in Northern Ireland negotiations while chief of staff for former British Prime Minister Tony Blair.
Israelis said talks with Hamas won't resolve the Middle East conflict.
"Hamas is conducting war against the citizens of Israel," said Ron Prosor, Israel's ambassador to Britain. "What do you say to people who say, 'Why don't you talk, try and talk, and not to shoot'? It sounds very good but the question is, at what stage do you do that?"
Celebrity Couples of 2014 [PHOTOS]