The election will decide whether the nation, with more than 40 million eligible voters, will see the return of liberals after five years of Lee's conservative government or hand power to a woman president for the first time.
The two candidates are the ruling Saenuri Party's Park Geun-hye, 60, daughter of former President Gen. Park Chung-nee, and the main opposition Democratic United Party's Moon Jae-in, a human rights lawyer and a former chief of staff to the late President Roh Moo-hyun, predecessor to outgoing President Lee.
Lee could not seek another term under the Constitution.
Yonhap said several pre-election surveys had shown the two running a neck-and-neck contest within the margin of error.
"Though it's cold today, I hope you will take part in the voting and open up a new era that every one of you has yearned for," Park said after casting her vote near her home in Seoul, Yonhap reported.
Moon, who voted in the southern port city of Busan, was quoted as saying: "The only way for the people to change the world is voting. The only way for the people to stand above political power is voting. ... If you have been unsatisfied over the last five years, please change the world with your votes."
Yonhap News said the candidates have divergent views on issues such as relations with the Communist North Korea, the free-trade agreement with the United States, corporate reforms and welfare policies.
The elections are being held a week after a belligerent North Korea, in its latest violation of U.N. Security Council resolution, fired off a long-range rocket to put a satellite in space that has raised concerns about whether the secretive and isolated country has progressed to developing intercontinental ballistic missiles capable of carrying nuclear weapons.
Yonhap said election officials were reporting a voter turnout of 11.6 percent 3 hours after polling began at 6 a.m. in freezing weather. The turnout was better than the 9.4 percent registered at the same time in the 2007 presidential poll, with the final turnout then at 63 percent.
South Korea's three major television stations were to announce results of their joint exit polls after polling closed at 6 p.m., although final official results will not be known until early Thursday.
Earlier in the campaign, software tycoon Ahn Cheol-soo withdrew from the race and threw his support to Moon, making the race tighter, Yonhap said. The report said as the contest heightened, accusations and counter-accusations of illegal electioneering flew.
"We expect a very close race, so it will probably take some time before a winner becomes evident," Chosun Ilbo newspaper quoted an official at the National Election Commission as saying.