Hunnam cited a busy schedule, though he was rumored to have gotten "cold feet" over the intense media scrutiny that would accompany the film franchise.
Now that the dust has settled, the 34-year-old actor has addressed the casting shake up in more detail.
"I felt like I had an interesting take on that character and felt like I could have done a good job of playing Christian Grey, otherwise I wouldn't have taken it on in the first place," he told Life & Style on Sunday.
"When you put the time into something like that and a character comes alive in your mind, it's heartbreaking not to be able to play him [...] It was definitely kind of heartbreaking having to say goodbye to that character and not bring it to life," he said.
Dornan, a Calvin Klein model who played a serial killer in The Fall, eventually got the lead. He'll star opposite Dakota Johnson when the film hits theaters next year.