After he and six members of his party took their ministerial oaths of office, Kejriwal told the crowd of thousands the "people of Delhi have taken the oath," the Press Trust of India reported.
Kejriwal's Aam Admi, or "Common Man" Party, won 28 of 70 seats in its first political effort to take control of India's largest city. He campaigned on a promise of a corruption-free government.
The AAP formed a coalition government with the Congress party, which lost control of New Delhi when it won only eight seats, the BBC reported.
"The oath we are taking today is not of Arvind Kejriwal and the other ministers but people of Delhi have taken the oath," he said. "The whole fight is not to make Arvind Kejriwal a chief minister but it is for a change of governance in Delhi.
"This is a huge victory for the people of Delhi,'' added the 45-year-old former revenue service officer as he addressed tens of thousands gathered at the ceremony. "The people of Delhi have shown that elections can be fought and won with honesty."
Speaking to the newly sworn ministers, he said: "We have not come here to become ministers. We were born to remove the arrogance of big parties. We should be wary that no other party has to take birth to dismantle us."
NBC reportedly holds celebs hostage to Jimmy Fallon's show
Kim Kardashian, Kanye West reportedly set wedding date