The rock star never got permission to build the two-bedroom, one-and-a-half story timber lodge, so the Rother District Council has advised him that if the issue is not resolved soon, the structure will have to come down, The Scotsman reported Wednesday.
Representatives for McCartney, 64, said he needed the lodge to maintain his privacy.
A statement on behalf of the former Beatle in a planning application to the council, said, "The owner has a requirement for privacy, seclusion and security that the buildings at Woodlands Farm cannot provide.
"This is due to the proximity of the public footpath that passes by the farmhouse and through the farmyard, and the activities connected with the working farm and machinery movements creating noise disturbance and safety issues."
"The lodge provides quiet, secluded and secure accommodation and is used only in lieu of the farmhouse, which will always be vacant when the lodge is occupied. The intention is not to increase the amount of residential property."
2014: The Year in Fashion [PHOTOS]
Polly Bergen, Emmy Award-winning actress, dies