She is the first woman to assume the post, Kyodo News reported.
"It's a special honor for me to be able to work to strengthen the close ties between our two great countries," Kennedy told reporters at the airport.
She also spoke of her father, who was assassinated Nov. 22, 1963.
"He had hoped to be the first U.S. president to visit Japan."
Kennedy, 55, will present her credentials to Emperor Akihito on Tuesday and will then begin her formal duties, the Jiji Press reported.
Kennedy is expected to deal with issues such as the realignment of U.S. military forces in Japan, including the contentious relocation of forces within Okinawa.
She will also likely focus on North Korea's missile and nuclear programs and the Trans Pacific Partnership free trade talks.
Kennedy, an attorney, played a role in President Barack Obama's 2008 and 2012 campaigns.