Over 10 years after the split, Carter confessed Duff is still the love of his life.
"Don't be that stupid douche that loses the love of your life forever.. Like me...," Carter wrote on Twitter after re-tweeting a photo of Duff's.
"I'll spend the rest of my life trying to better myself to get back to her. I don't care what ANY of you think," he continued.
"That tweet just gave me an anxiety attack," he added later. "People who have no idea who I am and/or what I've been through. If you're that interested watch my interviews or google me."
Celebrity Families of 2014 [PHOTOS]
Beyonce, Jay Z reportedly renewed their vows