"People are skeptical about incumbents," Frank said during a news conference in Newton, Mass. "There was also this: I don't like raising money."
He said his redrawn 4th Congressional District had too many new constituents, which he estimated at 325,000, more than he could serve in two years, the Boston Herald reported.
Frank said he also has had a "busy and stressful" four years because of tackling financial reform after the recession.
He said he was stepping down sooner than planned because of "anger" in the country and the reality that the behind-the-scenes work he did well "won't be constructive."
"I will miss this job and I will have some twinges of regret," Frank said. "But one of the advantages is I don't have to pretend to be nice to people I don't like."
Frank, former chairman of the House Financial Services Committee, was first elected to represent Massachusetts' 4th Congressional District in 1980.
While not revealing his plans, Frank, one of a handful of openly gay congressional members, said he wouldn't be a "historian or lobbyist."
When the Democrats were in power, Frank led the House Financial Services Committee and in 2010, helped guide the landmark financial regulation bill that bears his name and the name of the Senate sponsor, former Sen. Christopher Dodd, D-Conn.
The House of Representatives "will not be the same" without Frank, President Obama said in a statement.
"For over 30 years, Barney has been a fierce advocate for the people of Massachusetts and Americans everywhere who needed a voice," Obama said. "He has worked tirelessly on behalf of families and businesses and helped make housing more affordable. He has stood up for the rights of LGBT [lesbian, gay, bisexual and trangender] Americans and fought to end discrimination against them."
Rep. Maxine Waters, D-Calif., is in line for Frank's spot on the committee and could become the panel's chairwoman if Democrats retake the House, The Hill said. The California liberal is considered left of Frank
However, The Hill reported Democratic leaders may look to someone other than Waters, who is battling a long-running ethics investigation over whether she helped secure federal funding for a bank in which her husband owned stock and served as a board member.
2014: The Year in Music [PHOTOS]