The decision affects 22 Israeli communities, nine towns and villages near the Gaza and Egyptian borders and 13 Israeli communities near Israel's northern border, Israel Radio, Haaretz and The Jerusalem Post reported.
The decision will go into effect next month.
Nearly 100 soldiers guard the 22 communities in shifts, Haaretz said.
The army said the decision was made for operational reasons and not because of budget constraints. Evaluations conducted by the army concluded enhanced border security measures including electronic sensors, patrols and other components no longer require the presence of soldiers inside the communities, Israel's Channel 10 and Channel 2 reported Tuesday.
Soldiers will continue to guard West Bank settlements against terror attacks, the television reports said.
Shlomo Vaknin, in charge of security for the West Bank settlements, told The Jerusalem Post he has been notified the number of soldiers guarding settlements will be reduced.
The decision to remove soldiers from the communities was harshly criticized by residents in southern and northern Israel who will be affected by the move.
"We totally oppose this," Haim Yalin, head of the Eshkol Regional Council near the Gaza Strip, told The Jerusalem Post. "There is no replacement for soldiers who defend places like Kerem Hashalom and Nativ Ha'asara, and give residents a sense of security. When people see soldiers with their uniforms and flak jackets guarding, they sleep soundly."
2014: The Year in Fashion [PHOTOS]
Syrian Al Qaida group executes Lebanese soldier