Police said the gunmen riding a motorcycle shot and killed Qasim Aqlin, identified as an assistant security officer at the embassy, The Yemen Observer reported.
Aqlin died at the scene, officials said.
The security officer told the Observer the attackers had the "the hallmark of al-Qaida," adding six others "have been assassinated in this way."
The Yemeni national was shot and killed while going to work, CNN said.
The embassy saw violent protests in September when hundreds of demonstrators in front of the compound protested the U.S.-made video "Innocence of Muslims," which debased the Prophet Muhammad. At least four protesters died and more than 30 people were injured in clashes between security forces and demonstrators.
Aqlin's job was to coordinate embassy security between Yemeni police and embassy personnel, The New York Times reported. He was investigating the incident last month when protesters angry over an Internet video mocking the Prophet Mohammad breached the embassy's gate and vandalized the embassy.
Aqlin had applied for American citizenship, The Times reported. Responsibility was not immediately claimed for the killing.