Kovalchuk opened the scoring 5:09 into the game and made it 3-1 just 37 seconds into the third period, giving him 35 goals, two more than fellow All-Star Rick Nash of the Columbus Blue Jackets.
Stefan gave Atlanta the lead for good, 2-1, with 90 seconds left in the second, capitalizing on a shorthanded 2-on-1 with Shawn McEachern.
The Thrashers' penalty-killing was the key. They were perfect in seven shorthanded situations -- five in the second period.
Pasi Nurminen made 28 saves for Atlanta, which improved to 5
Mark Messier set up both goals for New York, which has won just five of its last 20 games overall (5-14