Bill Shorten, who has been the member of Parliament for the Melbourne suburb of Maribyrnong since 2007, beat Anthony Albanese in a tight faceoff, ABC news reported.
Shorten was secretary of the Australian Workers' Union for six years before entering Parliament where he served as Minister for Education and Minister for Workplace Relations.
After a month-long contest, Shorten, 46, received 52 percent of the combined vote of caucus and rank-and-file members.
Albanese, 50, is a former minister for transport and communications portfolios as well as a past deputy prime minister.
After his defeat, Albanese was quick to pledge his "total loyalty" to Shorten and took a swing Prime Minister Tony Abbott and his Liberal National Party coalition government.
"The honeymoon is over, baby ... for Tony Abbott and his [Liberal] team," he said.
"Bill will have the united Parliamentary Labor Party, as well as the membership, behind him over the coming years."
But Albanese also warned that Shorten "needs to run a disciplined show" if Labor is to defeat the Liberals.
"If anyone [in the Labor Party] steps out of line, he needs to discipline them and I will be there standing by him to assist in that process," he said.
About 30,000 votes were received from rank-and-file Labor Party members across the country -- the first time they have had a direct say in who leads the party, Radio Australia reported.
The leadership contest between Albanese and Shorten pitted union power against rank-and-file party members.
The Radio Australia report noted that Shorten played a pivotal role in the leadership instability of Labor's years in government that ended in a majority win for the Liberals under Tony Abbott on Sept. 7.
A general election soon after gave Gillard and Labor a slim minority government.
Earlier this year, Shorten then reversed his support to help return Rudd to the prime minister-ship.
But Rudd's battle against his old foe Gillard was to prove a fateful bloodletting for the party.
In last month's general election, Labor lost to the Liberal's alliance with several smaller parties.
The coalition defeated Labor 91 seats to 54 and Rudd immediately announced his resignation, prompting the leadership race.
Liberal Prime Minister Abbott has pledged to end the carbon tax on big business, stop the flow of thousands of illegal immigrants arriving mostly boats, balance the budget and begin construction of "the roads of the 21st century."
But Shorten especially is committed to keeping the carbon tax, saying that a failure to put a price on carbon pollution "merely delays problems for tomorrow's generation."
The Canberra Times reported Shorten won't budge on his pledge to keep the tax.
''On something as important as putting a price on carbon pollution, I've stated during the leadership campaign that I believe it is important to maintain a price on carbon pollution,'' he said.