Ottawa had scored on the power play in a team-record 13 consecutive games and owns the NHL's second-ranked unit with the man advantage.
Despite a sluggish start and no power-play goals in three chances during the opening period, the Senators got going in the second as Mike Fisher beat Martin Biron with a one-timer at 2:11 to forge a 1-1 tie.
Just over nine minutes later, White scored the go-ahead goal, his third in as many games.
Chris Gratton scored for the Sabres, who have lost six straight and are 0-9-2 in the last 11 games. The winless streak is one short of the club record, set during the 1991-92 season.
Gratton opened the scoring 2:54 into the game when he fired a hard shot from the bottom of the right faceoff circle past Lalime and under the crossbar. It was the only goal Buffalo got, despite an 11-6 edge in shots in the opening 20 minutes.
Ottawa tied it when Martin Havlat left a drop pass for Fisher and screened Biron, enabling the 22-year-old center to score his second goal of the season.
Hossa, who was stopped on a breakaway in the first period, made it 3-1 at 7:33. He scored off a perfect feed from 6-9 defenseman Zdeno Chara in the low slot.
Since outscoring foes, 12-2, in its first two games, Buffalo has just 24 goals while allowing 45.
After White's second goal, the Senators had a chance to make it 14 straight games with a power-play tally as Mair drew a major penalty and a game misconduct for intent to injure Chris Neil during a brawl.
But protecting a three-goal cushion, Ottawa took it easy and did not extend the streak.
Exploding whale video goes viral on Internet
Video of Victoria’s Secret models trying to 'twerk' hits Instagram