Both Gemba, a free-trade advocate, and Azumi are in their 40s.
Noda, a fiscal conservative succeeding Naoto Kan who resigned, had already appointed his close ally Osamu Fujimura, 61, as the powerful chief Cabinet secretary who becomes the government's top spokesman.
Noda's immediate challenges include a nuclear crisis at the Fukushima Daiichi nuclear plant triggered by a March 11 earthquake and tsunami.
He also must steer the recovery of the country's economy, the world's third largest, which was hit hard by the disaster, and maintain unity within his ruling Democratic Party of Japan.
Noda, 54, the sixth prime minister in five years, had been finance minister in the Kan Cabinet.
Among Fujimura's tasks as the chief Cabinet secretary will be to raise Noda's profile and promote his policies, Kyodo News reported.
In announcing Noda's Cabinet, Fujimura said, "I would like to build a foundation for cooperation between the ruling and opposition parties," The Wall Street Journal reported.
Azumi is reported to have useful connections among opposition parties, which could be useful for Noda because the upper house of Parliament is controlled by the opposition. Azumi is from Miyagi Prefecture, which was hit by the March 11 devastation.
Goshi Hosono, 40, retained his post as state minister in charge of dealing with the Fukushima nuclear crisis. He would be called upon to revamp the country's nuclear regulatory system, Kyodo said.
Tatsuo Hirano will continue as minister in charge of reconstruction. He is respected for his knowledge of agriculture and the economy.
Yoshio Hachiro, 63, was named the new industry minister. He favors nurturing businesses for building a stronger economy.
Late last month, Moody's rating agency lowered Japan's debt rating a notch to Aa3 from Aa2, saying the downgrade was "prompted by large budget deficits and the buildup in Japanese government debt since the 2009 global recession." The agency said the outlook, however, looks stable.
Japan's total debt is about 200 percent of its GDP.
"The March 11 earthquake and tsunami, and the subsequent disaster at the Fukushima Daiichi Nuclear Power Station, have delayed recovery from the 2009 global recession and aggravated deflationary conditions," Moody's said.