Two police officers also were wounded in the shooting at the New Castle County Courthouse in Wilmington, where the gunman also was killed.
David T. Matusiewicz, a former optometrist who had recently been released from federal prison after serving sentences for bank fraud and kidnapping his three daughters, was identified as the shooter, The (Wilmington) News Journal reported.
The (Wilmington) News Journal said Matusiewicz and ex-wife Christine Belford were involved in a bitter custody fight. The couple had been divorced since 2006 and his parental rights had been terminated. He had been seeking joint custody.
Matusiewicz kidnapped his young daughters in August 2007 when they were just 4, 6 and 7, taking them on an 18-month odyssey through Central and South America. They were found in 2009, along with Matusiewicz's mother, in a trailer in Nicaragua.
The identity of the second dead woman was not immediately released.
The two injured officers were described as a 50-year-old man who has been on the Capitol Police force for less than a year and a 42-year-old seven-year veteran.
It wasn't clear whether the gunman was killed by Capitol Police officers or killed himself, Delaware State Police Sgt. Paul Shavack said.
The injured officers were taken to a nearby hospital with non-life-threatening injuries, officials said.
Ralph Hilton told The News Journal he got a call from his sister, Victoria Warren, who was at the courthouse for jury duty.
Hilton said his sister told him, "They were shooting, they were shooting."
The shooting began about 8 a.m. when the gunman was stopped by Capitol Police at a security checkpoint inside the front door.
Police blocked off surrounding streets and nearby schools went on lockdown, WCAU-TV, Philadelphia, reported. While inside, visitors and courthouse employees ran for cover.
"I saw the gentleman and a bunch of ladies and people running out of the court building and I just hear all the bam, bam, bam, so I ran too," Irvin Neal, who was on his way into the courthouse, told WCAU.
Delaware Gov. Jack Markell, said in a statement, "Our thoughts and prayers right now are with the victims of this senseless violence and the Capitol police officers who put their lives on the line to protect the courts and public every day."
Ferris Wharton, a public defender and former chief deputy attorney general, said an employee in his office reported seeing a woman dead in the lobby, The News Journal said.
Witnesses told The News Journal the shooting began around 8 a.m. when the gunman was stopped at a security checkpoint inside the front door.
"I don't know what to say," Wharton, who worked in the Wilmington courthouses for three decades, told the newspaper. "I've never heard of anything like this happening in Delaware. It's a wild situation."
Sandra Autman, a deputy county clerk, said she and several other "hysterical staff" were in their mezzanine office, one floor below the lobby. She said employees heard at least five shots.
Ron Burgundy interviews Peyton Manning on SportsCenter
Man spent 15 hours in jail for plugging electric car into an outlet at a school