While Japan had only 550 soldiers in Iraq, the mission was the country's largest since World War II.
Koizumi notified leaders of the Liberal Democratic Party at a meeting Tuesday morning, The Washington Post reported. He said he would make a public announcement later in the day and declined to give a timetable for the pullout
"When the time comes, I will decide when will be the appropriate timing to have the ... troops return home safely after getting the understanding of other countries and the Iraqi government," Koizumi said.
On Monday, Iraqi Prime Minister Nouri al-Maliki announced that his security forces would be taking over in Muthana Province, where the Japanese have been based. Security in the province had been the work of British and Australian forces, while the Japanese worked on reconstruction.
Also on Tuesday, Australian officials announced their troops would be transferred from Muthana to the Syrian border.