The victim's son said the stolen money was compensation paid to his mother by Germany, Ynetnews reported.
Michael Goichberg, 89, told police he withdrew 1,600 euros ($2,088) from the bank Friday. The money is a regular stipend paid his wife, a Holocaust survivor from Ukraine.
Goichberg said two men came up behind him on a crowded street, said they would stab him if he made any noise and grabbed the money.
His son, Mark, told Ynet the thieves knew exactly where the money was.
"They didn't ask him for it, they just took it," Mark said.
Goichberg, who served in the Soviet army and was wounded, said he heard the siren marking Holocaust Remembrance Day while he was on his way to the bank and stood at attention.
Police have arrested two suspects.
NBC reportedly holds celebs hostage to Jimmy Fallon's show
Ray Liotta sues skin care company over use of likeness