Pakistan's Interior Ministry said the early morning assault on the mosque began after negotiations between the two sides broke down. Eight commandos have been killed so far during the military operation, CNN reported.
The ministry said Ghazi was killed as government forces and militants inside the mosque exchanged gunfire.
Militant students first took control of the mosque and an adjacent women's religious school last Tuesday in an effort to have Sharia (Islamic) law imposed in Islamabad.
While some 1,200 Islamic students have surrendered since, an unknown number of militants remained in the mosque holding women and children hostage.
Ghazi had previously said that 1,900 people were inside the compound and that more than 300 people had been killed during the week-long fighting.
A Pakistani army spokeswoman told CNN about 40 women and children escaped during Tuesday's raid.