According to State Department spokesman Mark Toner, Tillerson and Yang agreed to "address the threat that North Korea poses to regional stability."
The two sides also affirmed the importance of a "constructive bilateral relationship."
But the press release from the State Department did not go into the details of the issues that were discussed between the two officials.
Yang was Chinese foreign minister from 2007 to 2013.
The phone call follows a recent announcement Beijing would ban all coal exports from North Korea.
The measure was being implemented in compliance with United Nations Security Council sanctions resolutions, China has said.
But the move could also be a signal from Chinese President Xi Jinping to U.S. President Donald Trump that he has made his move and the United States, in turn, should do its part, according to The New York Times.
It is also not clear what effect, if any, a recent meeting between Tillerson and Chinese Foreign Minister Wang Yi had on the Chinese decision.
Tillerson and Wang had met on the sidelines of the G20 gathering of foreign ministers in Bonn, Germany last Friday, according to Yonhap news agency.
The top U.S. diplomat reportedly addressed the growing threat of North Korea nuclear weapons and missile programs during his first meeting with Wang. Tillerson may also have requested Beijing to rein in Pyongyang to safeguard regional stability.
Analysts have said North Korea should remain a top priority for Trump because of the multiple risks that are associated with the Kim Jong Un regime.
Former CIA Deputy Director Michael Morell said on "Face the Nation" on Sunday there are "three significant risks" with the regime: state collapse, nuclear weapons sales to an overseas interest and an intercontinental ballistic missile launch.
A North Korea launch aimed at the United States is of the "lowest risk," Morell said.