Chauffeur Thomas Heiser alleges in a Fort Lee Municipal Court affidavit that Burke, 47, punched him in the face after he asked his employer if he could leave work to celebrate his wife's birthday, the New York Post reported Sunday.
"He called me a piece of (expletive), (expletive), lazy mother (expletive), which wasn't unusual," Heiser alleges of Burke's response to his query. "He said, 'Give me the keys. You walk home.'"
Heiser alleges during the June 11 incident, he also had his head slammed with a car-trunk hood at the hands of Burke.
Burke, owner of restaurants like New York's David Burke Townhouse, previously appeared on the TV series "Iron Chef America."
Police said Burke allegedly admitted to insulting Heiser, but denied the chauffeur's assault allegations. If convicted on related charges, Burke is facing up to six months in jail.