The top-seeded Federer posted a 6-3, 2-6, 7-6 (7-4) victory against the sixth-seeded Ljubicic to nail down a spot in Saturday's semifinals.
World No. 1 Federer wasted three match points and came within two points of losing on Day 3, but the Swiss rebounded and drilled a backhand down the line to settle the third-set tiebreak after 2 hours, 17 minutes.
Ljubicic, who has appeared in eight finals this year, is making his Masters Cup debut. The Croat was the runner-up at the Paris Masters just two weeks ago and has won 21 of his last 25 matches, including titles in France and Austria.
Ljubicic, 26, leads the ATP with 30 indoor victories this season.
Federer, 24, is appearing in the Masters Cup for a fourth straight year.