The two actors appeared Saturday night at the Paley Center for Media in New York to discuss the show, which celebrated the 20th anniversary of its debut last month, The Wall Street Journal reported Sunday.
Anderson said she was told she won the role of FBI Special Agent Dana Scully on a Thursday and by Saturday she and Duchovny were in Vancouver filming the pilot.
Duchovny said the show's creator, Chris Carter, wanted him to wear swimming trunks in the episode.
"I said nobody swims in boardshorts," so he wore his own Speedo. "I have regretted it ever since."
Anderson said she and Duchovny were often in bad moods when they had promotional photo shoots for the series because of the intense filming schedule required to produce two dozen episodes per season.
The shoots were held on weekends and "it was all day and that's your one moment off," Duchovny added.
"We were doing interviews every week," Anderson added, saying they didn't realize the popularity of the series until the third season.
2014: The Year in Music [PHOTOS]