Police said a guard at Ambassador Wolfgang Dold's residence saw four people fire at the gate using AK-47 assault rifles, ekathimerini.com reported.
Bullets were found embedded in the steel gate, police said.
The incident was being investigated by an anti-terrorist squad, which had detained six people for questioning. Police said security video was being reviewed.
Greek Foreign Minister Evangelos Venizelos, who spoke with Dold, condemned "today's cowardly act of terrorism," the Foreign Ministry said in a statement.
The statement said the "sole and obvious target" of the attack was "Greece's image abroad just a few days before the start of the Hellenic presidency" of the European Union council.
The last time the German ambassador's residence was attacked was in May 1999 when members of the terrorist group November 17 launched a rocket at the building.