Brown scored from the 1-yard line in the first quarter and ran 14 yards for a score in the second for the Gators (1-0), who were playing without six players who had been suspended earlier in the week for violating team rules.
Jeff Driskel completed 17-of-22 passes for 153 yards, including a 4-yard TD to Gideon Ajagbe with 55 seconds left in the game.
Florida's defense held Toledo (0-1) to 205 yards of total offense and two field goals by Jeremiah Detmer.
Owens hit on 17-of-38 passes for 155 yards and was picked off once.
2014: The Year in Music [PHOTOS]