Former Irish President Mary Robinson, who traveled to North Korea with Carter and two other former Western leaders, said the delegation had a written message read to them "to be conveyed" to South Korean leaders, Yonhap reported.
In the message, Kim "specifically told us that he is prepared for a summit meeting directly with President Lee Myung-bak at any time to discuss any subject directly between the two heads of state," Carter said during a news conference. "Although we did not meet with the leader of North Korea, when we had already departed from our guest home, we were asked to come back to receive a personal message."
Robinson said the delegation, known as the Elders, was surprised at being asked to return to get the message that included "the possibility of a summit."
South Korea, which has said it is open to a summit with North Korea, hasn't responded to the proposal, the news agency reported.
Kim "sent word he is willing and the people of North Korea are willing to negotiate with South Korea or the United States" or other countries involved in the so-called six-party talks on denuclearization of the Korean Peninsula at any time and without pre-conditions, Carter said.
The six-party talks, dormant since 2008, include the Koreas, Japan, Russia, China and the United States.
The Elders is a group of former heads of state, brought together by former South African President Nelson Mandela in 2007 to support global peace and humanity.