The comedian said he playfully wrestled Cowell to the floor Friday when Cowell tried to halt an act he didn't like from performing.
“There were a couple called Que Sera, who sang Love Changes Everything, and he said, ‘Oh I hate this song.’ But we all wanted to hear it," Walliams said.
“So he tried to put his hand in the air to stop the song and we ended up wrestling. He said, ‘I’m a serious music mogul, people can’t see me wrestling on TV,' " Walliams added. “Simon is quite a serious person and he doesn't want me dragging him down to my level, as I learned at one point when we had a wrestle."