"There was pressure on me -- people wanted ministers from their regions," said Mustafa Abushagur, alleging members of Parliament representing contentious ethnic groups and divided regions failed to put aside provincial claims for the common good.
The Parliament, known as the General National Congress, voted 125 to 44 in favor of removing him, with 17 lawmakers abstaining and 14 no-shows.
With the dismissal, Libya now also lacks ministers of defense and interior, responsible for apprehending the killers of U.S. Ambassador Christopher Stevens, The New York Times reported.
The government has detained no suspects in the Sept. 11 attack on the U.S. Consulate in Benghazi, where Stevens and three other Americans were killed, the Times said. Leaders of Ansar al-Sharia, the main militia group that witnesses saw mounting the attack, are still at large.
"There is a power vacuum," Mohammed Ali Abdallah, a member of the National Front Party from Misurata who voted in support of Abushagur, told the Times.
"I think a lot of these militias will try to take advantage of this to exert their own authority," he said. "I think the ramifications will be quite bad."
Most of the Parliament is made up of regionally elected lawmakers. Libya's national parties -- the moderate National Forces Alliance and the Muslim Brotherhood's Islamist Justice and Construction Party -- failed to form working majorities, the British newspaper The Guardian reported.
The Parliament is led by Mohammed Magarief, who has been de facto head of state since August.
Abushagur -- who was deputy prime minister under interim post-Gadhafi leader Abdurrahim al-Keib, beginning in November 2011 -- had put forth 10 names for key ministerial posts Sunday shortly before the no-confidence vote.
"Because the country is facing challenges, I presented a mini-Cabinet," he said, explaining the proposed Cabinet was chosen without "geographical considerations."
He had until Sunday -- 25 days from his Sept. 12 appointment -- to form a Cabinet that had parliamentary approval.
Some cities, such as Benghazi, Misurata and Zawiyah, lost a lot of fighters during the battle to overthrow Moammar Gadhafi and demanded disproportional representation in the new government, Abushagur said.
One regional party demanded 11 seats and another nine, the BBC quoted him as saying.
Abushagur's offer last week of 29 Cabinet members was also rejected.
During Sunday's debate, about 1,000 protesters -- mostly from Zawiyah -- stormed the congress building in Tripoli calling for Abushagur's resignation.
The council now has three to four weeks to elect a new prime minister, the BBC reported.
The new government would run the country day to day until a new constitution is ratified.
Abushagur, 61, is a Libyan- and U.S.-educated electrical engineer who taught at the University of Alabama in Huntsville for 17 years. He also worked in the exiled opposition to Gadhafi.
His election as prime minister-designate followed a parliamentary vote from among eight nominees. Mahmoud Jibril, who was interim prime minister during the Libyan civil war, came in second.
Lawmakers speculated Sunday Jibril and the Islamist Justice and Construction Party would negotiate a new prime minister, the Times said.