Romney, who lost to Obama in the 2012 presidential race, told the Boston Herald that Robert Copeland should also apologize.
"The vile epithet used and confirmed by the commissioner has no place in our community," Romney said in a statement. "He should apologize and resign."
The controversy exploded after a Wolfeboro resident told a local newspaper about remarks she had overheard Copeland making in a restaurant. At a community meeting Thursday, most of the speakers demanded Copeland's resignation and some said they would organize a recall election if he does not step down.
Wolfeboro, which bills itself as "The Oldest Summer Resort in America," is a small town on Lake Winnipesaukee founded in the 18th century. Romney is one in a long list of famous summer visitors who include New Hampshire's last royal governor, TV host Jimmy Fallon and Princess Grace of Monaco.
Scott Brown, the former Republican Massachusetts senator now running in New Hampshire, also called for Copeland's departure. A spokeswoman said in an emailed statement that Brown believes Copeland's remarks were "reprehensible."