A tarp surrounded the Ramallah mausoleum where Arafat was buried and workers began removing the tombstone early Tuesday, CNN reported.
French authorities initiated an inquiry into the PLO leader's death this year after high levels of a radioactive material were found on some of his personal belongings.
Arafat's widow, Suha Arafat, lodged a formal legal complaint for murder. She told CNN she wanted her husband's body exhumed so investigators could be "100 percent sure" the radioactive substance was present.
Palestinian Authority President Mahmoud Abbas approved the exhumation of Arafat's body from the mausoleum, which is in the Palestinian presidential compound.
Arafat was 75 when he died in a Paris military hospital after suffering a brain hemorrhage and slipping into a coma.
Just before his death, Palestinian officials said he had a blood disorder and digestive problems. Suggestions that he had been poisoned had circulated at the time but then-Palestinian Foreign Minister Nabil Sha'ath said he "totally" ruled them out.
A Swiss doctor later said investigators had found high levels of toxic polonium-210 on some of Arafat's belongings, CNN said.
A body fluid stain indicated 180 mega-becquerels per liter of polonium-210; a typical sample would contain 5 mega-becquerels per liter, the doctor said. The fabric of Arafat's clothing, without body fluid, contained less than 10 mega-becquerels per liter. (A becquerel is a measurement of radioactivity.)