Officials said the baby panda, a male, was the first to be born at the zoo in 24 years, The Asahi Shimbun reported.
The cub's heart wasn't beating when zookeepers picked him up after he was found lying on his back on the belly of his mother, Shin Shin, at 7:30 a.m. Wednesday, officials said.
Zookeepers massaged his heart and tried other means of resuscitating him.
The cub was pronounced dead at 8:30 a.m.
Video footage and photos of Shin Shin and the baby panda had been shown nationwide on newscasts and in newspapers and crowds came to the zoo after the cub's birth.
The cub, born July 5, was the first born at the zoo through natural breeding. The Asahi Shimbun said natural breeding of the giant pandas, an endangered species, is difficult to achieve in captivity.
The cub was born to Shin Shin, 7, and Ri Ri, 6, both of which arrived in Japan in February 2011. The giant pandas came to Japan from the Conservation and Research Center for the Giant Panda in Sichuan province in China.
2014: The Year in Music [PHOTOS]