But teachers' unions said the decision, handed down Tuesday in Los Angeles by Superior Court Judge Ronald Treu, will do nothing to improve schools. Randi Weingartner, president of the American Federation of Teachers, called it "a sad day for public education."
Appeals could delay any changes in the system for years.
Treu ruled in favor of a group of nine students represented by the advocacy group Students Matter. He found that using seniority in layoffs, teacher tenure and giving teachers more protection from firing than other public employees violates the Equal Protection clause of the U.S. Constitution.
"The students who brought this lawsuit are, unfortunately, just nine out of millions of young people in America who are disadvantaged by laws, practices and systems that fail to identify and support our best teachers and match them with our neediest students," Duncan, who formerly headed the Chicago public schools, said. "Today's court decision is a mandate to fix these problems."
Duncan said the United States needs to have a "fair, thoughtful, practical and swift" discussion of tenure and other job protections for teachers.
Teachers in California have enjoyed more job protection than many of their colleagues across the country. The state grants tenure after two years, and Treu said that dismissing ineffective teachers is a cumbersome process that can take anywhere from two years to a decade and cost as much as $450,000.
"Given these facts, grossly ineffective teachers are being left in the classroom because school officials do not wish to go through the time and expense to investigate and prosecute these cases," Treu said.
The plaintiffs in Vergara v. California estimated that as many as 3 percent of the state's teachers are incompetent and said those teachers are far more likely to end up in low-income neighborhoods.
The California Teachers Association said it plans to appeal. Any changes in practice are unlikely while the legal battle continues.
Union President Dean Vogel called Treu's decision "deeply flawed."
"This lawsuit has nothing to do with what's best for kids, but was manufactured by a Silicon Valley millionaire and a corporate PR firm to undermine the teaching profession and push their agenda on our schools," Vogel said in a statement.
2014: The Year in Music [PHOTOS]