Police said Altaf Baba, the divisional commander of Pakistan-based Jaish-e-Mohammad, was killed Wednesday while his associate Mohammad Abbass Nengroo was shot to death Thursday, The Times of India reported.
The gun battle broke out Wednesday afternoon in the Rajpora area of the Pulwama district in southern Kashmir.
A foot patrol of 44 operatives of the specialist anti-terror force Rashtriya Rifles came under fire from militants, the report said.
A police spokesman said Baba the was killed in Wednesday's confrontation.
The body of Baba, also known Gazi Baba, was handed over to his family, police said.
Police later came under attack in the streets of Pulwama as news of the killings spread.
Five policemen were slightly injured when their vehicle was overturned during their attempt to escape an angry mob throwing stones and other objects. No serious injuries were reported.
Jaish-e-Mohammed -- Army of Mohammad -- was responsible for the 2001 attack on the Indian Parliament that left nine people dead, an action that put the organization on the radar of security forces around the world, The Times of India reported in February.
The group emerged out of Harkat-ul-Mujahedin shortly after the hijacking of an Indian Airlines plane from Kathmandu in December 1999. The hijackers demanded the release of three militant leaders, two of whom later became leading figures of Jaish-e-Mohammed.
The United States designated Jaish-e-Mohammed a foreign terrorist organization in December 2001, the Congressional Research Service Report for Congress said in its February 2004 document.
"This group is potentially a serious threat to stability in South Asia," the CRS report on foreign terrorist organizations said.
"Its ties to al-Qaida and the former [Afghanistan] Taliban regime make it a significant concern for the United States. Its murder of Daniel Pearl and targeting of U.S. facilities may indicate that JeM will continue to target other American citizens and interests in the future."
Pearl, 38, was a journalist with American and Israeli citizenship kidnapped by Pakistani militants and beheaded by al-Qaida in Pakistan in February 2002.
Pearl was working as the South Asia bureau chief of The Wall Street Journal, based in Mumbai, when he was kidnapped on a trip to Pakistan. He was investigating alleged links between shoe bomber Richard Reid and al-Qaida.
The CRS report said attacks by JeM exacerbate tensions between India and Pakistan, which remain deadlocked over the Kashmir issue.
Kashmir has been a flashpoint since the two countries were created and the area divided when the Britain gave up colonial control of the subcontinent in 1947.
Pakistan claims all the Kashmir region of India's Jammu and Kashmir state, which is around 60 percent Muslim -- India's only Muslim majority state.
Pakistan and India have fought several wars over the territory but agreed on a cease-fire line in 2003.
India continues to battle insurgents -- many of whom it claims are based in Pakistan -- who want the territory to be part of Pakistan.
Tensions in Kashmir, especially the summer capital Srinagar, have been high since February when Mohammed Afzal Guru, from Kashmir, was hanged in New Delhi for his alleged part in the 2001 Parliament attack.
His supporters in Srinagar demand the return of his body.