The freshman Republican said he planned to return to D.C. this week to finish out his term.
“I am committed to serving the 5th District to the best of my ability through this term," McAllister told the News-Star after making the decision Monday morning, "but I also have to take care of my family as we work together to repair and strengthen the relationship I damaged."
Kelly McAllister, who was to fly to Washington with the congressman Monday, stood by her embattled husband because "she knows it's going to be firestorm when I get there and she didn't want me to face it alone," he said.
"I'm behind him 100 percent," Kelly McAllister added.
Melissa Hixon Peacock, the aide who appears kissing Rep. McAllister in the leaked surveillance footage, was removed from the payroll.
McAllister won a special election last November to replace Rep. Rodney Alexander, who resigned to join Gov. Bobby Jindal's cabinet.
Several of the candidates who lost the special election in November could run to replace McAllister in the Louisiana 5th, considered a safe Republican district. State Sen. Neil Riser, who lost to McAllister in a runoff, is considered the mostly likely contender by local party officials.
2014: The Year in Music [PHOTOS]