Brust stole the ball and scored to break a 62-62 tie with 43 seconds to play in overtime.
Michigan (21-3, 8-3 Big Ten) had two field goal attempts in the closing seconds but could not convert.
Brust scored 14 points in the game to lead the Badgers (17-7, 8-3), who have won three consecutive games. Jared Berggren came away with 13 points and eight rebounds, and Sam Dekker scored 12 points.
Trey Burke scored 19 and Tim Hardaway Jr. scored 18 for the Wolverines, who shot 39.4 percent from the field and scored only one basket in overtime.