The victim, an older white male, was taken from the first floor clerk's office to a hospital immediately after the shooting, but was pronounced dead at the hospital, the Birmingham News reported.
FBI agents and Birmingham police were at the scene at the Hugo Black U.S. Federal Courthouse.
The newspaper said chaos erupted inside the courthouse when the shot was heard.
"I sat down and prayed," witness Peter Montgomery said. He said the victim "had blood pouring out of his head."
A release from the U.S. Marshals Service said a specially trained deputy U.S. marshal Medic rendered emergency aid to the man after the shooting.
The News said it was unsure how a gun was taken into the courthouse, as members of the clerk's staff are required to go through metal detectors like the public.
2014: The Year in Music [PHOTOS]