In the 2003 romantic comedy, Grant played the prime minister of Britain, while Thornton played the president of the United States.
Grant said he couldn't resist teasing his co-star when he heard Thornton suffered from numerous phobias, including European antique furniture and silverware, as well as images of Disraeli.
Grant recalled his interaction with Thornton when asked by reporters in New York last week how he felt about the silly, sinister ways Queen Victoria and scientist/writer Charles Darwin are portrayed as characters in his new animated movie, "The Pirates! Band of Misfits."
"Oh, that was a big plus. I hate those two. I've always hated those two," Grant joked.
"There are people that have fixations on historical characters," he said. "I remember doing 'Love Actually' with Billy Bob Thornton -- who, as you know, is unusual -- and he had a proper phobia about 19th century British Prime Minister Benjamin Disraeli. He's terrified of him. We had a set that was supposed to be 10 Downing Street and I found a picture of Disraeli and I used to come up and just push it in front of Billy Bob and he would break out in a sweat."
Thornton explained his aversion to Disraeli in a 2007 interview with Empire magazine.
"I was watching some old movie once -- I don't remember the name of it -- but Benjamin Disraeli was a character in it," Thornton said. "He was in this old, weird room, with old heavy drapes and there was just something about his hair. I thought, 'God knows what's going on with that hair.' I don't know what it was -- maybe some food accidentally got in there somehow. There was something about the way it swept out... I couldn't breathe too well."
"The Pirates! Band of Misfits" opens nationwide Friday. The 3D Aardman Animations stop-motion movie also features the voice talents of Imelda Staunton, Martin Freeman, Salma Hayek, Jeremy Piven and David Tennant. It is rated PG.
Celebrity Breakups and divorces of 2014 [PHOTOS]