Robert King, a State Department expert on North Korean human rights issues, had expected to visit North Korea soon but information from Pyongyang Sunday was that the invitation had been withdrawn, Voice of America reported.
It was not immediately clear why the invitation was pulled. A State Department official said the action could be related to upcoming military exercises between the United States and South Korea.
Whle North Korea turned away King, it did, on Monday, receive a delegation led by Donald Gregg, a former U.S. ambassador to Seoul, who was invited by the North Korean Foreign Ministry, VOA said. Lynn Turk, another ex-U.S. diplomat, said they were there "to build bridges" between the countries.
Neither Turk nor Gregg indicated whether Bae would be discussed, Voice of America said.
U.S. Secretary of State John Kerry will visit South Korea this week and is expected to discuss North Korea-related issues.
South Korea said military drills with the United States will begin later this month.
North Korea has threatened to back out of this month's planned family reunions of Koreans separated by the Korean conflict.
State Department spokeswoman Jen Psaki said the Obama administration was "deeply disappointed" with Pyongyang's decision and urged that Bae be granted "special amnesty and immediate release" as a humanitarian gesture meant to get him emergency medical care.
Bae, 45, was arrested in North Korea in 2012 on charges he was trying to use his Christian convictions to stir up a rebellion. He was in North Korea on business for his China-based company that arranges tours of the tightly controlled state.
He was transferred from a labor camp to a Pyongyang hospital in August for health reasons. On Friday, the State Department said Bae had been transferred from a hospital to a labor camp.