The 2010 Global Trends report said the developing countries' share of refugees is huge both in absolute terms and in relation to the size of their economies.
Pakistan (1.9 million), Iran (1.1 million) and Syria (1 million) have the largest refugee populations.
The report said Pakistan, in terms of economic impact, has 710 refugees for each dollar of its gross domestic product, followed by the Democratic Republic of the Congo (475) and Kenya (247).
By comparison, Germany has the largest refugee population among industrialized nations, with 594,000 people, or 17 refugees for each dollar of its GDP.
The refugees include 27.5 million people displaced within their own country by conflicts.
"Fears about supposed floods of refugees in industrialized countries are being vastly overblown or mistakenly conflated with issues of migration," said U.N. official Antonio Guterres.
"Meanwhile, it's poorer countries that are left having to pick up the burden."