The court voted 8-7 to strike down the ban, with the majority finding it violates the equal protection clause of the U.S. Constitution.
Michigan voters approved Proposal 2 -- which was called the Michigan Civil Rights Initiative -- by a vote of 58 percent to 42 percent in 2006.
State Attorney General Bill Schuette said he will appeal the ruling, the Detroit Free Press reported.
Proposal 2 "embodies the fundamental premise of what America is all about: equal opportunity under the law," Schuette said in a statement.
"Entrance to our great universities must be based upon merit," the statement said. "We are prepared to take the fight for quality, fairness and the rule of law to the U.S. Supreme Court."
Detroit attorney George Washington, who represented the plaintiffs in the court challenge of Proposal 2, called Thursday's ruling "a tremendous victory for students."
"It means that thousands of black students and Latino students will have the chance to go to college they never would have had," he said.
The U.S. Supreme Court has heard arguments this term on whether to ban the use of affirmative action in admitting students at the University of Texas.