Martin, 51, coached the Senators for 8 1/2 years.
"The sad part for me is not to be part of this great organization because I know they are going to win a (Stanley)Cup," Martin said. "I wish them the best and I'll regroup."
No replacement was named.
"In order to take the next step toward winning a Stanley Cup, we must select a new face, a new voice with a fresh approach and with new ideas," General Manager John Muckler said at a news conference.
The Senators made the playoffs in each of Martin's eight full seasons as coach. But they never reached the finals.
"It's like the oldest cliche in the world in our industry, you can't fire 29 people," Muckler said. "Is it right? Sometimes it's not, but I know one thing that's right. We needed a change."
Also fired were assistants Phil Myre and Don Jackson.