Kasay's game-winning kick came after New Orleans' defense stopped a 4th-and-inches running play by the Falcons at their own 30-yard line.
Atlanta's Michael Turner was stopped for no gain on the play, giving the Saints the ball already within field goal range.
Drew Brees finished with 322 yards and two touchdowns, connecting on 30-of-43 pass attempts for the Saints (7-3), who won their second straight game.
Marques Colston made eight grabs for 113 yards in the victory, which snapped a three-game winning streak for Atlanta (5-4).
Matt Ryan hurled a pair of TD passes en route to 351 yards, but it wasn't enough for the Falcons.
Turner compiled 96 yards on 22 carries and Harry Douglas made eight catches for 133 yards in the loss.
Celebrity Couples of 2014 [PHOTOS]