Cain hit a two-run shot off Addison Reed (3-2) to hand the White Sox their eighth loss in 11 games.
Cain also hit a solo homer in the fifth that put Kansas City ahead 3-2, but Dewayne Wise tied it again in the bottom half of the inning with a home run.
Both sides scored twice in the sixth to keep the contest deadlocked before Cain came through in the ninth.
His second home run came after Johnny Giavotella had opened the ninth with a single. Kansas City has defeated the White Sox in six straight meetings.
Kelvin Herrera (2-2) got the win by working the eighth inning. Greg Holland gave up a single, but also struck out the side in the ninth for his 12th save.
Caroline Berg Eriksen: Soccer player's wife triggers debate with post-birth selfie
Man behind Doritos Locos Tacos passed away on Thanksgiving