The package, addressed to Josef Ackermann, the Frankfurt bank's top executive, was intercepted Wednesday by police, CNN reported.
The bomb, which contained shrapnel, was detected in the bank's mail room, said New York Deputy Police Commissioner Paul Browne.
"The return address was listed as a European central bank, which would likely increase the chances of him opening it," said Browne, who said the incident in Germany prompted Deutsche Bank offices in New York to beef up their security.
The Frankfurt prosecutor's office said "initial investigations show that this was a functional letter bomb."
There has been no immediate claim of responsibility for sending the bomb, The New York Times reported, and police have not named any suspects or suspected groups.