Saleh traveled to the Saudi capital to sign the agreement negotiated by the Gulf Cooperation Council in hopes of quelling protests that have rocked the nation for most of the year.
"Today a new page opens in your history," Saudi King Abdallah told Yemenis attending the ceremony. Arab News said Abdallah urged all those concerned to honor the agreement, which ends Saleh's 33 years in office.
The agreement calls for the formation of a national unity government within 14 days and presidential elections within 90 days. Executive powers will pass immediately to Vice President Abd al-Rahman Mansur al-Hadi.
U.S. President Barack Obama issued a statement welcoming the agreement.
"For 10 months, the Yemeni people have courageously and steadfastly voiced their demands for change in cities across Yemen in the face of violence and extreme hardship. Today's agreement brings them a significant step closer to realizing their aspirations for a new beginning in Yemen," Obama said, pledging to stand by Yemen during the transition.
The agreement allows Saleh to keep his position until the elections and grants him immunity from prosecution, The New York Times reported.
Pro-democracy demonstrators for months have called for Saleh's resignation. In the past he has indicated a willingness to do so only to back out of signing similar agreements.
Opposition leaders and Yemen's allies have pressed Saleh to sign a deal, warning that the country was near collapse because of continued protests and bloody fighting among factions.
Saleh's surprise trip to Riyadh followed several days of negotiations between opposition leaders and the president's representatives brokered by a U.N. envoy, the Times said.
Yemeni opposition leaders were to go to Riyadh later Wednesday for the signing.
Previous agreements have been upended by violence in the Yemeni capital of Sanaa between government forces and defecting army units and tribal fighters loyal to Saleh's rivals. The Times said there were reports Wednesday of sporadic shelling in Hasaba, a district in northern Sanaa.