Chaloka Beyani, a U.N. special envoy for the rights of internally displaced persons, updated reporters on the situation in Georgia following a five-day visit last week.
He expressed gratitude to the government for inviting him but said he was frustrated he was unable to visit the disputed territories of South Ossetia or Abkhazia.
Georgian President Mikheil Saakashvili took his country to war with Russia in 2008 over claims to the breakaway republics of Abkhazia and South Ossetia.
Tensions rekindled briefly in early June when Russian security forces erected a security fence along the border with South Ossetia.
"I regret that the placement of a fence along the administrative boundary line of South Ossetia has the effect of depriving internally displaced persons and displacement affected communities of freedom of movement and livelihoods," Beyani said in a statement Monday from Geneva.
NATO Secretary-General Anders Fogh Rasmussen is scheduled to visit Georgia next week at the invitation of the government.
Georgia aspires to join NATO. It's the largest contributor of forces to the international mission in Afghanistan among non-NATO members.