With most of the ballots counted, Hollande had captured 28.59 percent of the vote to 27.09 percent for Sarkozy in Sunday's first round, Radio France Internationale reported Monday.
Sarkozy became the first sitting president to place second in the first round of voting, RFI said.
Far-right Front National candidate Marine Le Pen's 18.6 percent pushed her out of the running but her performance in the election will make her a force in the second round, France 24 said.
"Tonight is historic," Le Pen said. "We are the only opposition to the ultra-liberal, libertarian left-wing."
Hard left candidate Jean-Luc Melenchon received 11.11 percent of the vote after polls had suggested he would fare better.
Melenchon urged those who had voted for him to cast ballots for Hollande.
"I want to thank warmly the voters who, through their votes, have placed me in this position," Hollande told a crowd of supporters in Paris, CNN said. "This is an act of trust of confidence in my [positions] that I have presented to the French people."
Sarkozy said France is in "a time of crisis" and thanked voters for turning out, saying, "I know [their] worries, and I understand them."
Sarkozy challenged Hollande to three debates over the next two weeks but a Hollande adviser said the Socialist would agree to only one.
"There has always been one debate and there's no need for [that] to change," Aurelie Filippetti told CNN.
"The French people have the right to truth and clarity," Sarkozy said. "Everyone will be able to make their choice with full knowledge."