Rev. David King, 52, was removed as vicar after losing his appeal Tuesday. King also was given a four-year ban on serving as an Anglican priest, The Times of London reported.
King, who was married at the time, faced a tribunal set up under the 2003 Clergy Discipline Measure in November 2007 and the body ruled that he had engaged in an "improper, intimate and physical relationship" with a married mother-of-four.
The diocese of York released a statement Tuesday saying King has been "permanently removed" from his office as Vicar and will not be allowed to exercise his priestly ministry for four years.
The Bishop of Whitby, Right Rev. Robert Ladds, said the church is working to secure the services of a new vicar.