Ilya Kovalchuk had the other Atlanta goal and Kari Lehtonen made 29 saves as the Thrashers head into the All-Star break with a 27-15-8 record, good for first place in the NHL Southeast Division. The Rangers have lost four of five and are 23-21-4 at the break.
Havelid broke a 1-1 tie midway through the third period off a face-off when he cut toward the goal and backhanded the puck, which appeared to hit the stick of New York goaltender Henrik Lundqvist before going into the net.
Hossa had been stopped earlier in the game on a penalty shot and also on a breakaway but got some revenge with an empty-net goal with 24 seconds left.
Kovalchuk scored the game's first goal 4:56 into the contest with a wrist shot that Lundqvist got a glove on but couldn't stop. The Rangers tied the score at 1-1 when Jaromir Jagr beat Lehtonen on his glove side with a wrist shot about three minutes later.