The remarks came as protesters stormed a government building in Donetsk and a group of gunmen threatened a U.N. diplomat in Simferopol. Earlier, Russian diplomats refused direct talks with their Ukranian counterparts.
Tensions in Ukraine have been rising since the weekend when foreign troops, presumably Russian, moved into Crimea following the ouster of Ukranian President Victor Yanukovych.
Kerry urged "intense discussions" in coming days to normalize the situation in Ukraine.
"I don't believe any of us is served by greater or further confrontation," Kerry said, adding the United States will not allow the violation of Ukraine's territorial integrity "to go unanswered." He said the situation in Ukraine has "united the world in support of the Ukranian people."
Kerry noted U.S. Defense Secretary Chuck Hagel has taken steps to reassure NATO allies, including expanding the U.S. air presence in Poland.
"Russia made a choice and we have clearly stated we believe it is the wrong choice. ... Russia can now choose to de-escalate this situation," Kerry said.
In Washington, a bipartisan group of U.S. senators introduced a resolution condemning Russian aggression in Crimea, and calling for withdrawal of the troops and a negotiated settlement.
"Russia must immediately pull back its military and allow a diplomatic solution to advance in Crimea -- one that respects Ukrainian sovereignty but also the unique history and makeup of the region," said Sen. Dick Durbin, D-Ill., a co-sponsor of the resolution. "There is no reason that the situation in Ukraine needs to be manipulated in a new battle between the East and West."
In Simferopol, the Crimea's regional capital, Robert Serry, a former Dutch ambassador to Ukraine, was surrounded by armed men as he left a meeting, the New York Times reported. Jan Eliasson, a U.N. deputy secretary general, said Serry was able to call him, is in "good shape physically" and was "not kidnapped."
James Mates, a reporter for the British broadcaster ITV who was with Serry, said the diplomat had decided to end his trip to Crimea, something U.N. officials could not confirm.
CBS reported Kerry was unable to get Russian and Ukrainian diplomats to talk face-to-face in Paris. Kerry is insisting direct talks are necessary to resolve the crisis and was scheduled to meet with Russian diplomats later Wednesday, CBS said.
"All parties agreed today it is important to try to resolve these issues through dialogue," Kerry said, adding that "greater or further confrontation" will not help anyone involved in the conflict.
Protesters at a pro-Russian rally occupied the Donetsk regional administration building in eastern Ukraine, Interfax-Ukraine reported.
The protesters broke through a cordon of Ukrainian military and police then entered the building through the broken front doors and hung a Russian flag from a window, the report said.
The Ukraine troops fled, the news agency said.
A plurality of the residents of Donetsk are Russian-speaking Ukrainians and ethnic Russians.
In Moscow, meanwhile, Russian Foreign Minister Sergei Lavrov said a Feb. 21 agreement that created a coalition government in Ukraine is absolute and must be the basis for peace in the country, RIA Novosti reported.
"The most heated phase of this crisis began when, in essence, an armed overthrow of the government and the legitimately elected president took place in violation of Ukraine's constitution," Lavrov said during a trip to Madrid.
The European Union-brokered agreement would have allowed Yanukovych's pro-Russia government to remain in power until early elections, Lavrov said.
Yanukovych was impeached by Parliament a day after the agreement was signed, enabling the opposition to take control and appoint interim leadership.
In other news, the European Commission has agreed to approve an offer of $1.5 billion in emergency aid to Ukraine, $625 million more than a previous offer, EUobserver.com said.
During a visit to Kiev Tuesday, Kerry said Washington would contribute loan guarantees worth $1 billion.
The International Monetary Fund is expected to contribute as well, amid reports the total contribution in financial aid will be about $5.4 billion, EUobserver.com said.
The aid package is expected to be approved Thursday before European Union leaders meet in Brussels to discuss the standoff between Ukraine and Russia.
Ukraine's interim government says it will need $34 billion in the next two years.
Ukraine is on the verge of bankruptcy after Russia walked away from an agreement made with Yanukovych to buy $15 billion of Ukrainian government bonds and cut natural gas prices in return for his rejection of the EU's free trade and association agreement.
Yanukovych's decision in November led to anti-government protests that resulted in the Russian ally's ouster last month. Yanukovych, who maintains he is Ukraine's legitimate leader, fled to Russia, which multiple media sources said sent troops into the pro-Moscow Crimea region saying ethnic Russians and Russian speakers were in danger, a claim that hasn't been independently verified.
RIA Novosti reported thousands of unidentified troops, apparently under command from Moscow, took control of Ukrainian military bases across Crimea. Russian President Vladimir Putin has denied Russian troops were in Ukraine, describing the well-armed troops seen in vehicles bearing Russian military license plates as "local militia."
Sergei Aksyonov, the prime minister of the autonomous Crimea region, says his government won't negotiate with officials in Kiev, RIA Novosti reported.
Crimea is expected to vote this month on its status, with greater autonomy and secession as possible options.
Andriy Parubiy, Ukraine's secretary of National Security and Defense Council, praised Ukraine's soldiers and the patriotism shown by residents of the nation's southeastern regions, the Kiev Post reported. Parubiy said their actions saved the nation from Russian plans to invade parts of Ukraine beyond the Crimean region.