A poll conducted by CBS News published Sunday shows 82 percent of U.S. respondents think it is either too early to say the summit will achieve U.S. goals of North Korea denuclearization, or denuclearization simply won't happen.
There is also a partisan divide on President Donald Trump's performance on North Korea policy.
While two-thirds of Republicans said Trump has been successful in negotiations, a lower percentage of Democrats said they have confidence in the president.
The poll was published a day ahead of South Korean President Moon Jae-in's visit to Washington.
South Korean news service SP News reported Moon will meet with Trump to exchange views on North Korea before the summit between Trump and Kim in Singapore.
Moon is visiting the United States upon Trump's invitation, and the two sides exchanged a phone call on Saturday, Seoul time.
Trump initiated the call after Pyongyang's KCNA issued a statement from former negotiator Kim Kye Gwan suggesting North Korea could pull out of the U.S.-North Korea summit.
Trump may also want confirmation of a "private assurance" the North Korean leader gave to Moon regarding Pyongyang's intentions for peace.
Senate Republicans are wary of North Korea intentions following the statement last week.
"If they don't show up that means diplomacy has failed and President Trump is intent on calling the question on North Korea in his first term," Graham said. "It would be time to take American families and dependents out of South Korea."
Graham also said the United States does not want to collapse the regime.
"We're not trying to spread democracy to North Korea," Graham said.