The fire broke out at Akasaka, a Japanese restaurant in Wallington, just after 1 a.m., WABC-TV, New York, reported.
Gregory Barnas, 57, who began working at the Wallington Fire Department in 1972 as a junior firefighter, was cutting a hole in the roof of the building when he fell and went into cardiac arrest.
Bergen County Prosecutor John Molinelli said two Barnas' sons are also firefighters and were at the scene when their father fell, the cable TV news channel News 12 New Jersey reported.
Molinelli said said investigators believed the fire was started by an electrical fixture in the kitchen of the restaurant.
WCBC-TV reported the flag at the Wallington firehouse was lowered to half-staff in memory of Barnas.
"It's a sad day, it's a tragic day," Wallington Fire Commissioner Mark Tomko said. "Our department and I know our community is mourning the loss and our hearts go out to the family."
2014: The Year in Music [PHOTOS]