The meeting is being held at Sunnylands in Rancho Mirage, Calif., an estate now owned by The Annenberg Foundation where every president from Dwight Eisenhower on has been a guest. Officials said the plan was to have a less-scripted meeting in a less-formal setting than the White House.
Obama and Xi spoke of the importance of the U.S.-China relationship during a brief media availability.
"President Xi just took office in March," Obama said. "Our decision to meet so early, I think, signifies the importance of the U.S.-China relationship. It's important not only for the prosperity of our two countries and the security of our two countries, but it's also important for the Asia Pacific region and important for the world."
Xi remembered President Richard Nixon's historic trip to China in 1973.
"And in the more than 40 years since then, the China-U.S. relationship has gone through winds and rains and it made historical progress," he said. "And our two peoples and the people elsewhere in the world have reaped huge benefits from this."
The schedule includes talks Friday along with a working dinner, followed by more talks Saturday morning, the White House said. Xi and Obama were to have some time alone.
They are expected to talk about each other's economic policies -- the countries have the world's two largest economies -- as well as North Korea's pursuit of nuclear weapons, the Syrian civil war, China's human-rights record and anti-U.S. cyberattacks, the administration official said.
Obama is expected to ask Xi to take "responsibility" for attacks that originate in China, the official said.
"Countries have to take responsibility for what emanates from inside their borders," the official said.
U.S. officials have expressed alarm about reports China has been stealing U.S. military high-tech secrets, which they say could cripple U.S. defense systems. Other reports indicate hackers in China are targeting U.S. businesses.
Beijing denies it engages in cyber-espionage and has not acknowledged targeting U.S. companies or military sites. It is expected to argue the issue is overblown, analysts say.
Notable deaths of 2014 [PHOTOS]