The House Oversight and Government Reform Committee voted 23-17 along party lines to hold Holder in contempt for refusing to turn over documents associated with the ill-fated effort to follow the movement of guns across the U.S.-Mexico border to drug cartels.
The Washington Post reported the action now goes before the full House, where a vote was tentatively scheduled for next week.
Earlier in the day, President Barack Obama asserted executive privilege, officials said.
Committee Chairman Darrell Issa, R-Calif., told reporters he intended to move forward with the contempt vote. Holder tried and failed Tuesday to negotiate an agreement with congressional leaders that would have avoided the vote.
Issa had set a 10 a.m. Wednesday deadline for Holder to submit documents on the investigation into the anti-gun running campaign.
The Justice Department has already provided the House panel and the Senate Judiciary Committee with 7,000 pages of documents, and Holder said he offered more. But Issa said the department submitted 80,000 pages to its inspector general.
In the Fast and Furious operation, federal agents between 2009 and 2011 allowed suspected illegal gun buys to go through and attempted to track the guns to Mexican drug cartels. Most of the weapons disappeared, with some resurfacing when they were used in crimes -- including two guns involved in the killing of a U.S. border agent.
"The attorney general wants to trade a briefing and the promise of delivering some small, unspecified set of documents tomorrow for a free pass today," Sen. Charles Grassley, R-Iowa, said.