Gere and his business partner, Russell Hernandez ,had the 5-foot-high cedar fence constructed after locals complained they didn't want to have to look at the inn's parking lot, the New York Daily News reported Monday.

Bedford officials, citing town code, then told Gere and Hernandez the fence is too high and want it torn down.

"It's a lovely building, but the fence sticks out like a sore thumb," Hazel Nourse, head of Bedford's zoning board, said in the Daily News report.

"If they say you have to have a 4-foot fence, then that's what we'll do," Hernandez said.

Among celebrities who call Bedford home are Martha Stewart, Ralph Lauren and Glenn Close.