Party lawmakers voted 71-31 in support of Gillard to stay on as head of the party and thus as prime minister. Rudd, who resigned as foreign minister last week, pledged loyalty to her, the Australian Broadcasting Corp. reported.
The two had been rivals since Rudd was replaced as prime minister by Gillard in 2010. The party vote was called after Rudd resigned as foreign minister and announced he would challenge Gillard for the party and government leadership posts.
ABC reported Rudd promised to dedicate himself to "working fully" for Gillard's re-election in the next general election to be held by November 2013. After his defeat Monday, he also promised not to resign from the party.
The BBC reported Monday's result is a major boost for Gillard, who must now bring her party together.
Gillard is expected to reshuffle her Cabinet, including filling the foreign minister slot. Defense Minister Stephen Smith is seen as the front-runner, the BBC said.
Gillard in a speech to Labor Party legislators prior to the voting had talked about progress made in the past 18 months and what needs to be done to win the next election, CNN reported, quoting party officials.
Commenting on the party vote, Rudd congratulated Gillard.
"I accept fully the verdict of the caucus and I dedicate myself to working fully for her re-election as the prime minister of Australia -- and I will do so with my absolute ability dedicated to the task," ABC quoted him as saying.