Voters at 150 polling sites across Crimea and 50 within Sevastopol, a city with a separate administrative status, were asked on ballots printed in Ukrainian, Russia and Tatar whether to secede and request Russian annexation, or remain part of Ukraine, with expanded autonomy, the Russian news agency RIA Novosti said.
The Voice of Russia reported lines of voters showed up before polls opened at 8 a.m., local time, including Crimean Prime Minister Sergei Askyonov.
The official tally was expected to be released Monday.
Western leaders have threatened economic retaliation and visa bans against Russian authorities if Crimea is annexed by Russia after the referendum.
The White House issued a statement Sunday rejecting the referendum, noting it was contrary to the Ukrainian constitution and administered under threats of violence and intimidation by Russia. It called Russia's action in Crimea "dangerous and destabilizing," and reinforced the U.S. support of Ukraine.
Hackers attacked the official website of the secession referendum just before polls opened Sunday, RIA Novosti reported.
"Today at 1 a.m. our website was hit by a massive wave of modern denial of service attacks," officials in Crimea said in a statement.
The statement said authorities were able to trace the source of the attack to the University of Illinois at Urbana-Champaign.
Similar attacks were made on Russian government websites Friday.