Fukuda's decision came after a Japanese navy Aegis-equipped destroyer ran into a small fishing boat in the Pacific Ocean Tuesday, The Yomiuri Shimbun reported.
Some officials reportedly believe the crash showed the need for increased unity between the ministry's civilians, also known as "suits," and military personnel, or "uniforms," the report said.
"I think there are also systematic problems at the Defense Ministry. The organization must be reviewed from the bottom up," Fukuda said to Ishiba Friday.
Ishiba reportedly established a "reform promotion team" the same day and started considering how to blend civilian and uniformed personnel from all Japanese military offices.
"It's possible for suits and uniforms to coexist and support the Cabinet by splitting the ministry into three functions -- developing defense capacity, conducting actual operations and overseeing Diet deliberations and explanations," Ishiba said during a Defense Ministry reform meeting.