The network announced Friday it had fired the political commentator after less than a year on the air because he did not represent the values on which Current TV was founded.
Olbermann, in turn, said he planned to sue the network for wrongful dismissal.
He talked about his departure from Current TV on Tuesday's edition of "Late Show with David Letterman."
"Do they know what they're doing over there?" Letterman asked. "Because you know what you're doing."
"Thank you," Olbermann replied, adding, "I screwed up."
"I screwed up really big on this," he said. "I thought we could do this. It's my fault that it didn't succeed in the sense that I didn't think the whole thing through. I didn't say: 'You know, if you buy a $10 million chandelier, you should have a house to put it in. Just walking around with a $10 million chandelier isn't going to do anybody a lot of good, and it's not going to do any good to the chandelier.' And then it turned out we didn't have a lot to put the house on to put the chandelier in, or a building permit, and I, I should have known that. And it is my fault at heart."
"You're the chandelier?" Letterman asked as the audience laughed.
"I'm the chandelier," Olbermann said. "You are always pointing out how big my head is, so I think it's a suitable analogy."
Letterman told Olbermann he was "impressed" with how he handled the situation and said he knows him to be "a stand-up guy who's ready for a good scrape and will take the high road if there is a possibility of a high road anywhere in the world anymore."
"So for you to announce that the whole thing was your fault just by agreeing to go there, you're taking the blame for that?" Letterman said. "You got your money. That's all I care about, right?"
"Well, up to last Thursday I got my money," Olbermann said. "The nice judge will decide whether or not I get more of my money. But quite seriously, you know, in that situation, what you're thinking is, 'Oh, Lord, this is probably going to hit the water at some point.' But what do you do? You could bail out and say I'm getting out of this immediately,' and trust me, I was thinking about that as early as like last July. We'd been on the air about 10 days and they fired the guy who knew what he was doing who I worked for and I went, 'Uh-oh.'"
Olbermann said he kept trying because he felt a sense of duty to his audience and his staff, some of whom followed him from other jobs.
"They put their careers at risk for me, and I didn't even get a chance to say goodbye to them, let alone thank them for the work that they did with me, and I'm, you know, I'm so proud of them because the show editorially was never better, but I let them down because the thing didn't continue," Olbermann said.
Celebrity Families of 2014 [PHOTOS]