The man entered St. James Catholic Church in Killingly Wednesday afternoon through a front door that was supposed to be locked, WVIT, Hartford, Conn., reported.
A class of third-grade students practicing for mass saw the man from only a few feet away, said the pastor, The Rev. Tracy Caffrey.
"It was very scary, they were definitely distraught," Caffrey said.
Some parishioners also saw the man, the pastor said.
"A teacher told the man to leave and for the students to go back to class," said the Rev. John O'Neill.
Police arrested Gary Pohronezny of Brooklyn. He was charged with disorderly conduct, risk of injury to minors, and interfering with police.
The suspect was taken to a hospital for a psychological examination.