They described the agreement during a photo session with Tokyo journalists before their bilateral meeting, CNN reported.
During the appearance with the media, Pompeo said he wanted "a fully coordinated, unified view of how to proceed, which will be what is needed if we are going to be successful in denuclearizing North Korea."
"I would like to have thorough coordination with you on our respective policies towards North Korea," Abe responded.
The visit to Japan on Saturday included a meeting with Abe and Japanese Foreign Minister Taro Kono.
While Pompeo headed off for the Asia tour Friday, he said he hoped to set up a location and time for the next summit between U.S. President Donald Trump and Kim.
The second summit between them could take place in Geneva, Switzerland, or Vienna, according to Japanese press reports.
North and South Korean leaders said that the North will permanently close its missile test site and dismantle its main nuclear facility if the United States takes corresponding measures under a deal struck at the inter-Korean summit between Kim and South Korean President Moon Jai-in last month.
Pompeo is expected to meet Moon after his visit to North Korea on Sunday.
He is scheduled to fly to China on Monday.