Two security officials said militant leaders had fled Shaqra town in Abyan province, CNN reported.
The Defense Ministry said most of the town was under government control and that militants have been surrounded by troops, who gave them a deadline to surrender.
The raid comes three days after Yemeni government troops seized control of the towns of Jaar and Zinjibar from militants.
In Azzan, an al-Qaida-controlled town in neighboring Shabwa province where militants were trying to regroup, 17 militants were killed Thursday in clashes with troops, local security officials said.
The Yemeni government increased its offensive against al-Qaida two months ago.
Yemeni President Abdurabu Hadi has called fighting the terror network the nation's top priority.