"As the economy improves, the demand for gold as a financial hedge declines more than the fundamental demand for gold jewelry increases," The New York Times quoted Daniel Arbess, a partner at Perella Weinberg Partners, as saying.
In an interview with The South China Morning Post, a news outlet in Hong Kong, billionaire investor George Soros said last week that, "Gold was destroyed as a safe haven, proved to be unsafe."
"Because of the disappointment, most people are reducing their holdings of gold," Soros said.
Gold's drop is partly a result of the bust and boom cycle in equity markets, the Times said.
Its value soared after the financial crisis of 2008. Gold is bought frequently as a hedge against losses in other markets and as a hedge against inflation.
The equity market is currently at a post-recession peak and the Federal Reserve's accommodating monetary policy has not sparked the inflation that some expected. As such, gold is less popular and the price is plummeting.
The price of gold is down 17 percent from its 2011 peak, dropping on Wednesday to $1,558 per troy ounce on the Comex division of the New York Mercantile Exchange.
That means the Federal Reserve Bank of New York's gold reserves have lost $75 billion of its value. The United States Bullion Depository, in Fort Knox, Ky., has seen the value of its bullion drop by $50 billion, the Times said.
In the long-term, however, gold is still doing well. From 1999 to 2013, gold's value has jumped 515 percent, a better return than the average equity portfolio, the Times said.