Cowell was asked on Tuesday's edition of the radio show "On Air with Ryan Seacrest" if he intends to be present when girlfriend Lauren Silverman gives birth to his first child.
"Are you out of your mind?" Cowell quipped. "There are certain things you shouldn't see and that is one of them."
But E! News said the statement was just an example of Cowell's dry sense of humor.
"Don't believe everything Simon says about not being in the delivery room. He jokes around a lot," one of Silverman's close friends told E! News. "I'm sure he was just trying to be funny. He's very committed to being there for the birth."
NBC reportedly holds celebs hostage to Jimmy Fallon's show
Millions of Getty images now available for free via embed tool