In the one-for-one deal, Dilfer, a 13-year veteran who quarterbacked the Baltimore Ravens to a Super Bowl win in 2001, will provide a veteran presence behind Alex Smith, last year's No. 1 overall draft pick.
Dorsey, 25, has eight touchdown passes and 11 interceptions in 10 starts in three years out of Miami (Fla.). He was a 7th-round draft pick by the 49ers in 2003.
"Trent Dilfer was instrumental in terms of the making over of our football team last year," Cleveland General Manager Phil Savage said on the Browns' Web site. "At this point, to add Ken Dorsey and to give Trent an opportunity to go back home to California is a win-win for all parties."
Dilfer was a sixth overall pick by Tampa Bay in 1994 and has 106 touchdown passes and 117 interceptions.