Vajpayee submitted his resignation to Indian President A.P.J. Abdul Kalam.
"We will sit in the opposition," said Venkiah Naidu, chief of Vajpayee's Bharatiya Janata Party.
The 22-party NDA was drubbed by opposition Congress Party and its allies.
Congress has won 200 seats of the 490 results declared so far. The Leftist parties, with 56 seats, have pledged support to a Congress-led government.
The NDA has won 190 seats. Results from 53 more seats are yet to be announced.
There are 543 elected members in the lower but powerful house of the Indian parliament. Only 539 results will be reported Thursday.
Congress Party chief, Italian-born Sonia Gandhi, is likely to stake her claim to form the next government.
"Of course, we are going to stake our claim today to form the next government. It will be a Congress-led government," senior party leader and political secretary to Congress President, Ahmed Patel, told reporters.
India's marathon election concluded Monday, in which 350 million people cast their votes.
Teacher apologizes for showing sexual image of herself in class
Dennis Rodman pledges to end trips to North Korea