Pettine, 47, has never before been a head coach past the high school level. He served as defensive coordinator for the New York Jets in 2009-12 and last season for the Bills.
The defenses he led finished each season in the Top 10 in the NFL in total defense.
"We are thrilled to announce Mike Pettine as the new head coach of the Cleveland Browns," said Browns owner Jimmy Haslam. "Mike is the epitome of what we want the Browns to be -- tough, aggressive and innovative -- with a blue-collar, team-first mentality.
"He knows what's necessary to beat teams in the AFC North. Most importantly, Mike has repeatedly shown the ability to lead his players to consistent improvement and success, clearly what we are striving for as he leads the Cleveland Browns moving forward."
Pettine replaces Rob Chudzinski, who was fired after one season, during the Browns were 4-12. Only the Houston Texans (2-14) and Washington Redskins (3-13) had worse records, although four other teams also won only four games last season.
Pettine is the 17th coach in franchise history, which began with Paul Brown in 1946, and the seventh since 1999.