There had been rumors about Rudd's removal from Prime Minister Julia Gillard's government, but his resignation effectively stopped any such effort, the Australian Broadcasting Corp. reported.
Rudd, a former prime minister who was replaced by Gillard in 2010, called a late night news conference in Washington to make his announcement.
Rudd said public attacks on him from Gillard supporters showed he no longer had the confidence of the prime minister, the network said.
"The simple truth is that I cannot continue to serve as foreign minister if I do not have Prime Minister Gillard's support," he said.
"I therefore believe that the only honorable thing and the only honorable course of action, is for me to resign. And I do so with a genuinely heavy heart and after much personal reflection," the ABC reported.
Rudd also was quoted as saying: "The truth is the Australian people regard this whole affair as little better than a soap opera, and they are right, and under current circumstances I won't be part of it."
He said the overriding question for his party colleagues is to decide "who is best placed to defeat Tony Abbott at the next election." Abbott is the leader of opposition in Parliament.
CNN reported tensions had been building up between Rudd and Gillard lately.
In his announcement, Rudd said he would return to Brisbane Friday to consult with his family and colleagues about his future, the ABC reported.
"There's no way I would ever be party to a stealth attack on a sitting prime minister elected by the people," Rudd said.
'Happy' fan videos make Pharrell cry [VIDEO]
Pregnant Mila Kunis wins 'Best Villain' at MTV Movie Awards