The gunmen stormed a Human Rights Ministry office, seizing several officials Thursday, China's state-run Xinhua News Agency reported.
Security forces sealed off the surrounding area, also home to the Interior and Transport ministries, the BBC reported.
The number of hostages and whether there were any casualties was not reported.
No organization has claimed responsibility for the incident.
Elsewhere in Baghdad, bombings near a market and a restaurant killed six people and wounded 20 others Thursday, several hours after car bombs exploded in several commercial districts of the capital city, MENA reported.
Iraq has experienced an uptick in violence in recent months. The United Nations estimated 7,818 civilians and 1,050 security force members died in 2013.