Second-ranked McIlroy had set himself up for the top ranking when he staged a furious rally to best world No. 3 Lee Westwood in Sunday morning's semifinal.
He needed only to down Mahan in the afternoon final to claim the top spot, but instead fell victim to the American, who earned $1.4 million for his fourth career PGA win.
Mahan will now likely move to No. 9 in Monday's new world rankings. He reached the championship round by eliminating Mark Wilson 2 & 1 in Sunday's other semifinal.
In McIlroy's victory over Westwood, the Northern Ireland native overcame a three-hole deficit after four holes, winning four of the next five to charge back into a 1-up lead. McIlroy moved 3-up with a birdie at 13, taking advantage of an errant Westwood tee shot.
The latter drained a 20-foot eagle putt at the par-four 15th, but later stumbled on No. 17 and conceded the stirring comeback win to McIlroy.