More than 780 people were killed when the Rana Plaza factory collapsed last month outside of Dhaka.
A five-member U.N. working group on human rights and business said clothing brands that have operations in Bangladesh have a responsibility to ensure working conditions are safe.
"International clothing supply chains are increasingly complex and addressing systemic issues is not an easy task but the scale of the efforts must be commensurate with the challenge," working group director Pavel Sulyandziga said in a statement.
Retailer Walmart said safety is "an industry-wide challenge in Bangladesh." Gap Inc. said it too was concerned about safety factors in Bangladesh.
The United Nations estimates that at least 3 million Bangladeshis are employed in the garment industry.
"We strongly urge international clothing brands sourcing from Bangladesh to address human rights risks in their supply chains with the involvement of workers, other relevant stakeholders, and human rights experts, and to share publicly what they are doing to mitigate their risks," said Sulyandziga.