The 35-year-old English actor discussed his short-lived but highly-publicized romance with the 27-year-old American singer in the March issue of GQ.
"Taylor is an amazing woman. She's generous and kind and lovely, and we had the best time," he told the magazine. "Of course it was real."
"I only know the woman I met. She's incredible," he said. "[But] a relationship in the limelight... A relationship always takes work. A relationship in the limelight takes work. And it's not just the limelight. It's everything else."
Hiddleston and Swift were first linked in June following the singer's split from DJ Calvin Harris. The actor was seen the next month in a "I Heart T.S." tank top, fueling speculation the relationship was a publicity stunt.
"The truth is, it was the Fourth of July and a public holiday and we were playing a game and I slipped and hurt my back. And I wanted to protect the graze from the sun and said, 'Does anyone have a T-shirt?' And one of her friends said, 'I've got this.' And we all laughed about it. It was a joke," Hiddleston explained.
"I have to be so psychologically strong about not letting other people's interpretations about my life affect my life. A relationship exists between two people. We will always know what it was," he said. "Nobody had the context for that story.
"I just, I was surprised. I was surprised that it got so much attention," the star admitted. "The tank top became an emblem of this thing."
Hiddleston and Swift split in September after three months of dating. Sources told E! News at the time that the couple disagreed about putting their relationship in the spotlight.
"Taylor felt he wanted to take the relationship to be too public and she was not happy about it and thought it was too much publicity," an insider said.
Hiddleston is known for playing Loki in Thor and other movies in the Marvel cinematic universe. He will reprise the role in Thor: Ragnarok, and will also star in the King Kong prequel film Kong: Skull Island.