On the other hand, that might be more easily said than done, the report concedes.
"For the foreseeable future, there is simply not enough capacity on the balance sheets of U.S. banks to allow a reliance on depository institutions as the sole source of liquidity for the mortgage market," a report commissioned by the Bipartisan Policy Center said.
The report was written by a group that included president of the American Bankers Association Frank Keating, a former governor of Oklahoma, former Housing Secretary and former Republican Sen. Mel Martinez of Florida and other panelists, including former Senate Democratic Leader George Mitchell and former Housing Secretary Henry Cisneros, who served under President Bill Clinton.
The New York Times reported Friday the report recommends phasing out the Federal Home Loan Mortgage Corp., known as Freddie Mac, and the Federal National Mortgage Association, known as Fannie Mae, but replacing them with a similar institution that purchases loans, but concentrates its focus on loans for low-income borrowers.
"Given the size of the market and capital constraints on lenders, the secondary market for mortgage-backed securities must continue to play a critical role in providing mortgage liquidity," the report said.
Through Fannie Mae, Freddie Mac and the Federal Housing Administration, the government now backs about 90 percent of all new mortgages, a record that was prompted by the financial crisis of 2008 and 2009.
Martinez said he envisioned that dropping substantially with the FHA backing just 15 percent of new mortgages and the Fannie Mae and Freddie Mac replacement backing 35 percent to 45 percent more.
The goal would be to have the private sector guarantee 40 percent to 50 percent of new mortgages, Martinez said.
That would be, more or less, like the good old days. In the 1980s, banks held about 75 percent of their loans, keeping the profits but also accepting the risks. That might be a worthy goal to shoot for, Martinez said.
2014: The Year in Fashion [PHOTOS]
GM recalls 221,000 Cadillacs and Impalas