Savage has spent the past two seasons as director of player personnel with the Baltimore Ravens.
Cleveland struggled to a 4-12 season in 2004 and Coach Butch Davis resigned in the midst of the campaign.
"The Cleveland Browns got better today," said Ozzie Newsome, vice president of the Ravens. "It just makes our jobs tougher."
Savage's first job in the NFL came as a scout for the Browns before they moved to Baltimore. He became director of scouting with the Ravens after the move and then was promoted to player personnel director in 2003.
Ohio crash that killed two caught on camera [VIDEO]
Wisconsin business offering 'therapeutic cuddling' forced to close