Damon's attire at the taping of Friday's show prompted host Letterman to joke, "(I'm) gonna have to get a dress code for the guests."
"Now, I know you're a football fan. I know you're not a New Orleans Saints fan," Letterman said. "What's the deal here?"
"I finished a couple months ago making a film with Anthony Mackie," Damon explained. "The great -- you know, the actor in 'The Hurt Locker' -- great, great actor -- and he's a big, big Saints fan. And we like to talk. So we made some bets. ... And then the Saints won the Super Bowl."
Asked what Mackie would have had to do if the Saints had not won, Damon said: "Had the Patriots won the Super Bowl, (Mackie) was going to have to put a Tom Brady jersey on. ... But the best part was he was going to go on 'The View' and no matter what they asked him, he was going to have to try to steer the conversation back toward Tom Brady's greatness."
Damon stars in the current release "The Green Zone." He also will compete for the best supporting actor Oscar Sunday for his work in "Invictus."
Celebrity Couples of 2014 [PHOTOS]