The Nationals let a four-run lead get away when the Braves scored four times in the seventh, but Morse hit the first pitch of the eighth inning from Chad Durbin (3-1) over the wall in right for the game-winner.
Washington won for the third time in four games, building its early lead on an RBI single by starting pitcher Ross Detwiler in the second, a two-run base hit by Ian Desmond in the third and a solo homer from Jesus Flores in the fourth.
Atlanta's four-run rally was finished off by a two-run shot by Andrelton Simmons.
The win went to Sean Burnett (1-1), who recorded the final out of the seventh and worked the eighth as well.
Tyler Clippard issued a one-out walk in the ninth before claiming his 13th save.