Military officials told ABC biblical references have been removed from about a third of its Trijicon rifle scopes, but these are being used on training bases, not in combat.
Critics say continued use of the "Jesus rifles" violates constitutional guarantees of separation of church and state and puts U.S. troops at greater risk.
"Shame on (the military) forever for their impossibly slow, plodding and utterly backwards plan for correction of this national and international disgrace," said Mikey Weinstein of the Military Religious Freedom Foundation.
"It's wrong, it violates the Constitution, it violates a number of federal laws," said Weinstein, an attorney and former Air Force officer. "It allows the mujahedin, the Taliban, al-Qaida and the insurrectionists and jihadists to claim they're being shot by 'Jesus rifles.'"
Weinstein said troops in Afghanistan have complained the Gospel inscriptions put them at higher risk.
The U.S. military bans "proselytizing of any faith, religion or practice."
The Defense Department said in January it would alter existing scopes with the help of modification kits supplied by Trijicon, which had promised to stop placing inscriptions on new scopes for the military.
2014: NFL Cheerleaders [PHOTOS]