The blasts hit the entrance to the shrine of revered 8th century saint, Abdullah Shah Ghazi, as people were coming out of Thursday evening prayers and gathering to distribute food to the poor, the BBC reported.
"I saw this boy with long hair. He was coming into the shrine when he blew himself up," one witness said.
No one has claimed responsibility for the attack as yet, the Los Angeles Times reported.
Pakistan President Asif Ali Zardari said he condemned the bombing "in the strongest possible terms."
"The relentless attacks on ordinary Pakistani citizens by those who want to impose an extremist mindset and lifestyle upon our country will not deter our government," said one of Zardari's top aides, Farahnaz Ispahani. "We remain committed to fighting these murderers and expelling them from our land."
Militant Islamists regard the Sufi discipline of Islam to be heretical, the Times said.
The Sufi community in Karachi has declared three days of mourning.
All Sufi holy shrines will be closed and security will be tightened, the BBC said.