Abe Karpen and Portman were cast as a married couple in "New York I Love You," an anthology of 12 short stories of love in New York City, The New York Daily News reported.

"I have my kids in religious schools and the rabbi called me over yesterday and said in order for me to keep my kids in the school I have to do what they tell me and back out," Karpen said.

A friend told the News that Karpen, his wife and children had gone out of town for the weekend to get away from the uproar.

Jan Korbelin, the movie's executive producer, called Karen "a great ambassador for his faith."

"We are very sorry that this has created a problem for him personally and for the community," Korbelin added.

Karpen followed the tenets of the Hasidic movement during the filming, refusing, for example, to hold Portman's hand.