The soldiers were holed up in barracks near the Antananarivo airport and announced they had taken control of the island nation two days earlier, the BBC reported.
Some of the rebels were behind a coup that brought President Andry Rajoelina to power last year and the unrest came on the day the country was voting on a new constitution.
The officers claiming control of the country include Col. Charles Andrianasoavina, who last year backed Rajoelina's move to take power after months of violence and political turmoil.
Observers said the group has lost faith in Rajoelina.
Defense Minister Gen. Lucien Rakotoarimasy said the evacuation wasn't ordered in preparation for an attack. But soldiers and vehicles were deployed along a road near the army base.
If a new constitution is passed, it would let Rajoelina stay in power long enough to organize a new election. It would also lower the age limit for presidential candidates from 40 to 35, meaning Rajoelina, 36, could remain as president, the Voice of America said.