Bangkok and Phnom Penh have been at a standoff around the 900-year-old temple in the Dangrek Mountains on the Thai-Cambodia border for several years and several military confrontations.
The International Court of Justice ruled in 1962 that the temple -- since 2008 a World Heritage site -- was on Cambodian land.
However, some access to the mountaintop site passes through Thai territory, a route that Thai troops occasionally seal off.
The recent agreement to withdraw troops is "the first step" in following the International Court of Justice's order in July last year, a report in The Bangkok Post said.
The court, which sits in The Hague, agreed on a vote of 11-5 that Thailand and Cambodia should immediately withdraw troops from the disputed area and for the area to become a temporary demilitarized zone.
The ICJ will make its final ruling on ownership likely late next year, the Post said.
Thailand's troop withdrawal and deployment of two companies of border patrol police officers were overseen by Defense Minister Sukumpol Suwanatat and Army Chief Prayuth Chan-ocha.
The border patrol will monitor nearly 7 square miles declared a demilitarized zone by the ICJ and of which just less than 2 square miles are disputed.
The Post quoted Sukumpol as saying he hoped Cambodia will adhere to the ICJ ruling.
"If they are gentlemen, they must honor the agreement," he said in response to reports that Cambodian soldiers remain in the disputed area posing as civilians.
Sukumpol also said that the troop pullback was an indication that Indonesia had lost territory in the disagreement.
"We are not at a disadvantage," he said.
Earlier this week Cambodia pulled back nearly 500 troops and replaced them with 255 police.
"It's an appropriate time to implement the ICJ's verdict," Cambodian Defense Minister Tea Banh said during the exit ceremony on the Cambodian side.
But Tea also warned "full peace has not been ensured," the Post reported.
Fighting has flared in the area within the past several years, notably in October 2008 when two Cambodian troops died and seven Thai troops were wounded in an hourlong gun battle.
The clashes have been condemned by the United Nations and the Association of Southeast Asian Nations, to which Thailand and Cambodia belong.
Indonesian Foreign Minister Marty Natalegawa is in Cambodia this week as part of mediation efforts by the government of Indonesia, which also is a member of ASEAN.
2014: NFL Cheerleaders [PHOTOS]