Jan. 9 (UPI) -- Downton Abbey's fourth season debuted on PBS last Sunday to record-breaking numbers, but many fans were still disappointed by the absence of the show's original leading man, Matthew Crawley, who died suddenly at the end of last season.
"A lot of people were quite upset by that," Stevens told Today hosts Willie Geist and Savannah Guthrie, who said his unexpected death left her reeling.
"I spend a lot of my time apologizing," he said. "Yeah, I get a lot of, 'My wife was very upset by this. You have to apologize to my wife!' Then they call them over. 'Katherine! Come over here. This is the young man that upset you.' 'Say sorry!' You have to apologize, in public."
Stevens, who said similar exchanges happen "almost every day," said it was "difficult" but "exciting" to make the decision to leave the show.
"It's a testament to how much people love the show, how much people love the character. In a way it was nice, but it has upset a lot of people, so I'm sorry," he added.
Meanwhile, Downton's executive producer Gareth Neame has shut down reports that the popular period drama will end after its fifth season.
"That was totally invented by the British press, totally made up. Completely untrue," Neame told Zap2It Thursday.
Stevens has moved onto other projects. His film Summer in February debuts in select theaters January 17.