Given there's been one inspector general's report and three hearings so far into the IRS' treatment of conservative organizations, the GOP lawmakers said calling for the Justice Department to act should be a last resort, The Hill reported Friday.
"When I can't do my job because I lack the authority or cooperation, I'll seek additional remedies," House Oversight Chairman Darrell Issa, R-Calif., told reporters Thursday.
"There will be more hearings coming," said Rep. Charles Boustany, R-La., chairman of a House Ways and Means oversight subcommittee and an early critic of the IRS' handling of Tea Party movement groups.
"I think it's premature," Boustany said of the prospect a special prosecutor.
GOP lawmakers told The Hill progress is being made on the IRS case without a special prosecutor. They also have expressed concern about the prospect of Attorney General Eric Holder -- who's had a rough relationship with congressional Republicans -- naming a special prosecutor to investigate the tax agency.
Some Republicans said they believe a special prosecutor eventually be necessary.
"I'm certainly not dismissing that," Rep. Jim Jordan of Ohio said. "It remains an option, but right now we're just getting started."
Rep. James Lankford, R-Okla., told The Hill the treasury inspector general who outlined the IRS' treatment of conservative groups made it clear IRS employees were reluctant.
"So now how do you get those answers?" Lankford asked.
"As long as you have access to grand jury power and subpoena power, yes, that's the way you investigate crimes," said Rep. Trey Gowdy, R-S.C., a former prosecutor. "Congress is not well-equipped to investigate crimes."
Notable deaths of 2014 [PHOTOS]