The swearing-in ceremony had barely concluded before Abbott's office issued a news release announcing that three Cabinet secretaries -- Don Russell of Department of Innovation, Industry Science and Research, Blair Comley of the Resources, Energy and Tourism Department and Andrew Metcalfe of the Agriculture Department -- had been fired, the Australian Broadcasting Corp. reported.
The release said Treasury Secretary Martin Parkinson would step down next year.
"Each of these secretaries has made a substantial contribution to public life in Australia and I wish them well for the future," Abbott said in the statement.
Abbott and other members of his government were sworn in by Governor-General Quentin Bryce at Government House in Canberra.
Abbott also announced that AusAID, Australia's overseas aid agency, will be moved to the Department of Foreign Affairs and Trade so diplomacy and aid delivery would be "more closely aligned."
The department also will handle overseas promotion of Australia as a tourist destination.
Archie Law, executive director of charity ActionAid Australia, charged that the oversight change for AusAID will see the aid budget used to promote Australia's national interests.
"The government's short-sighted decision to integrate AusAID into DFAT will have massive and devastating effects on Australia's aid program and on the people living in poverty that the program supports," Law said in a statement.
Other changes include moving the Customs and Border Protection Service from the Attorney General's office to the purview of the new Immigration Minister Scott Morrison. The Immigration Department no longer will be responsible for settlement services for refugees and migrants, which shift to Social Services.
Abbott's coalition government promised to cut 12,000 public service jobs over the next four years through attrition with a goal of saving $5 billion, the Australian broadcaster said.
Abbott characterized Wednesday as "an action day," saying he would begin work to repeal the carbon pricing system "immediately," and to bring back temporary protection visas for asylum-seekers who arrive by boat.
Abbott promised his government would "strive to govern for all Australians.
"We won't forget those who are often marginalized; people with disabilities, Indigenous people and women struggling to combine career and family," he said. "We will do our best not to leave anyone behind."
Celebrity Breakups and divorces of 2014 [PHOTOS]