The score with 5:35 remaining in the game handed a close victory to the Cardinals (5-0).
Perry's TD was his second of the game, which was played in a persistent heavy rain. He ran the ball 22 times for 119 yards overall. Teddy Bridgewater completed 9-for-13 passes for 88 yards, a touchdown and one interception for Louisville.
The Golden Eagles (0-4) were led by Desmond Johnson, who picked up 87 yards on 13 carries.