The price per gallon averages $3.76 nationwide and analysts expect it to reach $4 by Memorial Day, USA Today reported Friday.
About 500 people were surveyed by Gallup.
While a huge majority said they wanted immediate action, 31 percent said they believe the price of gasoline may be out of the hands of President Barack Obama or Congress.
"We'll continue to see plenty of blame attributed to parties if we see prices go higher," said Tom Kloza, chief analyst for the Oil Price Information Service. "But there is very little a president can do to impact gas or crude oil prices over the short term."
Crude oil prices are expected to stay around $107 a barrel, the Organization of Petroleum Exporting Countries' latest monthly forecast said.
In explaining its findings, OPEC said, "The weak pace of growth in [economies in Europe] is negatively affecting oil demand."
Europe's debt crisis "along with higher oil prices has resulted in considerable uncertainties for future oil demand for the rest of the year," OPEC said.
Poll participants said gas prices have yet to hit a point that would make them consider changing their lifestyles or spending habits, Gallup reported.
Most people who responded to the poll said gas prices would have to hit $5.30 per gallon before they made significant cutbacks.
The Gallup poll was conducted nationwide Monday and Tuesday and has a margin of error of plus or minus 5 percentage points.