Brown's scores were key, coming in the fourth quarter.
Stafford played three quarters, completing 13-of-17 passes for 141 yards as the Lions improve to 2-1 in exhibition play.
Detroit also got a rushing TD from Kevin Smith and Chris Houston returned a fumble 14 yards for another score. Bryant Johnson caught a 7-yard scoring pass.
Browns quarterback Jake Delhomme was 20-of-25 for 152 yards and a touchdown in the first half. Peyton Hillis scored a rushing TD and Lawrence Vickers caught a scoring pass as Cleveland slipped to 1-2.
The Browns also capitalized on a fumble with Eric Wright running 44 yards with a dropped ball. Phil Dawson added two field goals.
Browns defensive back Nick Sorensen was removed from the field on a stretcher after suffering an unknown injury during kickoff coverage in the second quarter.