The town of Radda, about 100 miles southeast of Sanaa, the capital, had been occupied for almost two weeks, CNN reported. Militants opened the jail, occupied government offices and mosques, and said they would impose Shariah law.
Residents said they fear another occupation.
"They were here for 12 days. They controlled everything and the government did nothing to help us," Ali al-Shuqaidi told CNN. "What guarantees do the people have that this won't happen again? We can't live normally if we have no government assurances."
Yahya al-Arasi, an aide to Vice President Abduraba Hadi, said the prisoner release came after four days of negotiations. He said the occupiers originally demanded 16 prisoners.
One of the released prisoners was the brother of the occupiers' leader.