Trailing 4-2 heading into the final period, the Senators began their comeback 3:11 into the frame when Colin Greening fired a shot from the left boards that hit a stick and deflected into the net.
Then with time running out, Kuba put Ottawa ahead with a wrist shot that beat a screened Bryzgalov. Nick Foligno added an empty-netter to ice Ottawa's sixth third-period comeback win this season.
Craig Anderson stopped 33-of-37 shots to earn the victory.
Matt Reed notched a goal and an assist for Philadelphia. Bryzgalov finished with 32 stops for the Flyers, who split a home-and-home series with the Senators.