He will be replaced on an interim basis by Ron Rolston, coach of Buffalo's American Hockey League affiliate in Rochester, N.Y.
Ruff, 53, was a player and captain of the Sabres and had coached the team since 1997. The Sabres reached the playoffs eight times under his guidance and made it to the Stanley Cup Finals once -- when they lost in six games to Dallas in 1999.
Buffalo is in last place in the Northeast Division with a 6-10-1 record. Ruff recorded a 556-442 career record with 78 times and 83 overtime losses.
"The hockey world knows how I and the entire Buffalo Sabres organization feel about Lindy Ruff, not only as a coach, but also as a person," Sabres owner Terry Pegula said in a statement. "His qualities have made this decision very difficult. I personally want Lindy to know that he can consider me a friend always."
Buffalo has not won a post-season series since 2007. Ruff won the Jack Adams Award as the NHL's top coach in 2006.
The only current coach in any of the four major North American pro sports to have coached longer than Ruff is Gregg Popovich of the NBA's San Antonio Spurs.