Justice Anthony Kennedy indicated the act, known as DOMA, violated states' rights and four other judges indicated they considered it a gay-rights issue, a tweet by Scotusblog.com said.
"#scotus 80% likely to strike down #doma," the tweet said "J [Justice] Kennedy suggests it violates states' rights; 4 other Justices see as gay rights."
Before getting to the merits of the case, the court spent about an hour discussing whether the matter was properly brought before it, The Washington Post reported. The Obama administration has said that it would not defend the law, and lower courts have ruled DOMA unconstitutional but the administration has said it would enforce the law until the Supreme Court rules.
During Wednesday's oral arguments, Justice Antonin Scalia commented on the contradictory positions, saying it was a "new world" when the attorney general could decide a law is unconstitutional but still enforce it, Post said.
Kennedy, considered a potential swing vote in the case, called it a "questionable practice."
Chief Justice John Roberts criticized the administration's decision, saying he didn't understand why Obama "doesn't have the courage of his convictions" and refuse to apply the statute.
Deputy Solicitor General Sri Srinivasan said Obama was trying to show respect for Congress by enforcing the law until the high court could rule on its constitutionality.
Arguments focused on one section of the 1996 act that states a range of federal benefits and considerations for married couples apply only to heterosexual unions.
Kennedy told the advocate defending the law it did promote "uniformity" in federal law, noting that there were 1,100 references to marriage in the federal code, and that the definition of a married person is "intertwined with daily life." Kennedy questioned whether the federal government may impose its view of marriage, which has "always thought to be" the domain of the state.
Nine states and the District of Columbia permit same-sex couples to marry.
Paul D. Clement, representing Republican House leaders who are defending the law, said Congress wasn't discriminating but keeping hands off experiments by the states on same-sex marriage.
The law does not punish states for allowing such unions, but allows the federal government to decide how it chooses to allocate its benefits, he said.
Justice Ruth Bader Ginsburg said federal rules that apply to married couples are so enveloping that in states allowing same-sex marriages, married same-sex couples can't get the federal benefits available to heterosexual couples, which creates "a skim milk marriage."
In the underlying case, Edith Schlain Windsor, an 84-year-old New Yorker, married her same-sex partner of 40 years, Thea Spyer, in 2007 in Canada. When Spyer died in 2009 of multiple sclerosis, she left her estate to Windsor.
As executor of Spyer's estate, Windsor paid approximately $363,000 in federal estate taxes, but filed a refund claim under a federal statute that says "property that passes from a decedent to a surviving spouse may generally pass free of federal estate taxes." The Internal Revenue Service denied the claim on the ground that Windsor is not a "spouse" within the meaning of DOMA Section 3 and thus not a "surviving spouse" within the meaning of the statute.
A federal appeals court ruled in her favor.
The justices heard arguments Tuesday on California's Proposition 8, a referendum that stripped same-sex couples of the right to marry. California voters approved Proposition 8, the California Marriage Protection Act, in 2008 with slightly more than 52 percent for, nearly 48 percent against but a federal judge declared Prop 8 unconstitutional and a three-judge appeals court panel in San Francisco agreed.