Graham, one of eight senators in a bipartisan group negotiating immigration reform, said Thursday family visas should be limited to spouses and minor children, the Los Angeles Times reported.
The newspaper said it was unclear how many of the senators working with Graham on the bill agree with his position.
Neither Graham nor the other senators involved would respond to questions Friday, the Times said.
"There is no final agreement yet, so I'd caution you to not read too much into reports about what may or may not have been agreed to," said Alex Conant, a spokesman for Sen. Marco Rubio, R-Fla., another member of the group.
Under current policy, there is no cap on visas for spouses and minor children, and about 65,000 siblings and 46,800 adult children receive visas annually. The number of visas for skilled workers and immigrants prepared to invest in the United States is limited to 140,000 a year.
Rep. Michael Honda, D-Calif., called family reunification a "line in the sand" Friday. He said ending family preferences would affect Asian immigrants more than others and would "would cause tremendous hardship and heartache for thousands of U.S. citizens and their loved ones."