The agreement comes amid continued clashes on the high seas around large island groupings between mainly China and its neighbors including the Philippines, Vietnam and Malaysia.
This week China slammed Manila over a Filipino politician's visit to a disputed Spratly island.
The China-ASEAN agreement, signed at the 44th ASEAN Foreign Ministers meeting in Bali, was welcomed by Chinese Assistant Foreign Minister Liu Zhenmin.
"This is an important milestone document on the cooperation among China and ASEAN countries," Liu told reporters at the meeting. "And we have a bright future and we are looking forward to future cooperation."
A communique from China and ASEAN stressed the importance of maintaining peace and stability in the South China Sea and "the continued exercise of self-restraint by all parties concerned."
The communique noted the proposal of the Philippines for having senior officials of ASEAN work with maritime legal experts to study ways of resolving the escalating disputes over islands and atolls in the South China Sea. Their proposals will be presented to the 19th ASEAN Summit, starting Sept. 24 in Bali.
ASEAN and China have agreed to establish more formal -- but general -- guidelines for conducting discussions on maritime issues but haven't laid out specific legal, bilateral or multilateral routes for tackling the thorny issue of territorial claims.
The deal is notable for its lack of detail, said analysts. The common factor in all South China Sea disputes is China. Beijing, which is beefing up its maritime patrols, lays claim to all the island groupings, even those much closer to the shores of the Philippines, Malaysia, Brunei and Vietnam.
The Spratly Islands -- the largest group -- lie off the southwest coast of the Philippines as well as Brunei and Malaysia. Ownership of the Spratly Islands is the most difficult of all the territorial claims because of the number of claimants, including Vietnam and Taiwan.
Further north, off the west coast of the Philippines, lies the Scarborough Shoal, disputed between China and the Philippines. Meanwhile, Vietnam and China are fighting for sovereignty over the Paracels, a group of islands south of China's Hainan Island province and off the east coast of Vietnam.
Solving the territorial claims is becoming a higher priority for ASEAN because of the increasing importance of natural resources to nations. Ownership of the islands -- most uninhabited -- means ownership of oil and natural gas resources on the seabed and fishing rights.
In early June, the Philippines government dismissed China's territorial claim to the Reed Bank -- or the Recto Bank, as Manila calls it -- which is around 80 nautical miles from Palawan province but around 500 nautical miles from mainland China.
"The administration has always asserted that it will dismiss out of hand any claim to what are considered integral parts of Philippine territory, such as the Recto Bank in western Palawan," Philippines presidential spokesman Edwin Lacierda said.
"Concerning our defense posture, the administration is determined to improve the capabilities of our military and coast guard to enable the effective patrol and protection of our national territory and exclusive economic zone."
Earlier this month, the Philippines ruled out any joint exploration with other claimant countries in the Reed Bank.
"The Reed Bank is not part of the Spratlys," Philippines Foreign Secretary Albert del Rosario told reporters. "What is ours is ours."
The decision follows alleged threatening behavior by Chinese naval vessels toward several Philippines resource research ships around the Reed Bank in March.
Vietnam recently said it will do everything in it power to protect its coastal waters after China allegedly interfered with two of Hanoi's seismic survey vessels 80 miles off its south-central coast -- well within its territorial waters and around 370 miles south of China's Hainan Island.
China rejected Hanoi's claim that its ships had cut the cables. Beijing countered saying the Vietnamese ships owned by the state oil and gas company PetroVietnam were operating in Chinese waters.
In June, U.S. Defense Secretary Robert Gates said in Singapore that the United States will increase its military activity in the area, even though the number of U.S. service personnel stationed in Asia may drop.
At the same time the United States will seek to boost military cooperation with Asian nations, including more joint military exercises, Gates said in a speech to the 10th Shangri-La Dialogue, an annual Asian security summit hosted by the International Institute for Strategic Studies.