Japanese television network NHK reported the prime minister met with Archbishop Paul Gallagher, the top Vatican diplomat, on Tuesday.
Abe asked Gallagher whether Pope Francis could make a Hiroshima visit in order to realize a world without nuclear weapons.
Japan has been increasingly calling for a world without weapons of mass destruction since it observed the 71st anniversary of the Hiroshima atomic bombing on Aug. 6.
"If the pope says a prayer for bomb victims, it will be a great force for the realization of a world without nuclear weapons," Abe said, according to NHK.
The archbishop said he would convey the message to Pope Francis.
Japan is wary of growing North Korea nuclear threats. Abe has previously spoken out on the dangers of war, while referring to the Hiroshima bombing as an event that must never be repeated.
Abe said in 2016 Japan will continue to stand by its commitment to nuclear nonproliferation.
The prime minister is scheduled to meet with U.S. President Donald Trump, who recently came under criticism for travel restrictions targeting nationals of seven Muslim-majority countries.
Abe said the new policy was being monitored but that he "would like to refrain from commenting" on U.S. policy, the Asahi Shimbun reported Wednesday.
Abe and Trump are likely to discuss trade negotiation during their upcoming summit.