The sketch, from 2000, featured Walken as a famous music producer that keeps asking Ferrell to keep adding more cowbell to a song that is being recorded.
Ferrell said on Thursday that years after the sketch, he visited Walken after the actor had performed in a play.
"I say hello to him backstage and he's like, 'You know, you've ruined my life,'" Ferrell said before he started to mimic Walken
"'People during the curtain call bring cowbells and ring them. The other day I went for an Italian food lunch and the waiter asked if I wanted more cowbell,'" the comedian continued
"From The Deer Hunter, to Pulp Fiction to 'More Cowbell.' That's all he gets now," the 52-year-old said.
Ferrell also played a round of Whisper Challenge with Fallon with the pair taking turns trying to guess what each person is saying while wearing noise-canceling headphones.
Ferrell went off-script and gave long stories instead of reading off one phrase. The film star also knew beforehand what phrase Fallon was going to read while he wore the headphones.