Hazare, a devoted Mahatma Gandhi follower, has been fasting since Tuesday. He had refused to leave jail until the coalition government of Prime Minister Manmohan Singh met his demands.
After his release, he arrived in New Delhi's sprawling Ramlila Grounds with rain falling and thousands following in a huge procession.
"Victory to mother India," shouted Hazare, seeming none the worse from his three days of fasting. He declared the anti-corruption fight would continue whether "I live or not."
His departure from the jail came after two days of negotiations with the government. Under an agreement, Hazare will undertake a fast for 15 days at the Ramlila Grounds subject to certain conditions. One is not to hold a fast-unto-death and to stop the fast if necessary for health reasons, a Times of India report said.
Crowds of supporters across the country, young and old, whose ranks have been swelling since Hazare was taken into protective custody Tuesday prior to the start of his hunger strike, had kept vigil at New Delhi's Tihar Jail
As soon as Hazare left the jail, they followed, jamming the streets along the way.
Police and security forces took extraordinary care to protect Hazare and to ensure peace.
The swelling Hazare movement comes as Singh's Congress Party-led coalition government confronts a growing number of huge corruption scandals and allegations of high-level graft, and the economy faces serious problems, including high inflation and rising food prices.
The prime minister, personally, has not been tarnished by the allegations.
On Wednesday, Singh said Hazare may be inspired by high ideals but the "path he has chosen … is totally misconceived and fraught with grave consequences for our parliamentary democracy."
Various media organizations have attacked the government's handling of the Hazare campaign, calling attention to what has been described as its total capitulation since the man's arrest, including the decision to release him from jail.
A banner front-page headline in the Times of India, said: "People March, Govt. Crawls" along.
Hazare said after 64 years of India's independence from British colonial rule "we still haven't achieved complete freedom," the Times of India reported.
Hazare, not much known around the country until recently, is pressing for a tougher "Jan Lokpal," or citizens' ombudsman bill against graft than the one before Parliament placed by the Singh government. His supporters say the government's bill is too weak.
Critics of his protest tactics say they will undermine India's democracy.