Researchers at Georgetown University in Washington found that at the most competitive colleges only 14 percent of students come from the lower 50 percent of families by income, The New York Times reported.
In addition, a comparison of low-income enrollment at elite colleges reveals wide disparities.
A student at Vassar, for example, is three times as likely to receive a need-based Pell Grant as one at Washington University in St. Louis.
"It's a question of how serious you are about it," said Catharine Bond Hill, president of Vassar.
Hill said "shame on you" to colleges with multibillion-dollar endowments and numerous tax exemptions that recruit a small percentage of low-income students.
At Vassar, Amherst College and Emory University, 22 percent of undergraduates in the 2010-2011 class received Pell Grants that go mostly to students whose families earn less than $30,000 a year.
One main factor is simply getting low-income students to apply, say officials of groups that work with poorer students.
Justin Bieber crashes Drake Bell's album release party
Rosie O'Donnell unveils nearly 50-pound weight loss