A video of the indiscretion surfaced Monday and since then, the staffer, Melissa Hixon Peacock, has been taken off the payroll.
Though he acknowledges he's "fallen short," the congressman said he doesn't plan to resign "unless there is an outcry for me not to serve."
His former competition for the congressional seat, Johnson, has issued that outcry.
"This is another embarrassment to Louisiana," Johnson said. "I think he needs to resign, and I don't think he can be effective."
Celebrity Breakups and divorces of 2014 [PHOTOS]