Rauf's brother-in-law and four Pakistanis were also killed when their car filled with ammunition was destroyed. Rauf was designated as the deputy leader in "Khorasan," an antique name for Afghanistan, by the Islamic State.
The National Directorate of Security in Afghanistan released a statement confirming Rauf's death. NATO officials have yet to confirm.
Rauf was captured by the U.S. military in 2001 and spent six years in Guantanamo Bay before being released to Afghan authorities.
Taliban leader Mullah Omar and Rauf reportedly had a falling out recently.
President Barack Obama condemned the Islamic State last Thursday, calling it a "brutal, vicious death cult."