The German shepherd, chained in the family's garage, apparently lunged at the officer, The Atlanta Journal-Constitution reported Wednesday.
Anthony Currie said the officer also pointed his gun at him and told him to put his hands up.
"I said, 'Why (did) you shoot my dog?' And he said, 'Well, I'll blow your brains out,'" Currie told the Journal-Constitution.
A DeKalb County police supervisor sent to the scene said the officer made an error and went to the wrong house.
An investigation determined that the address the officer was locking was across the street from the Currie home.
An internal police investigation is under way.
Exploding whale video goes viral on Internet
Benedict Cumberbatch's dramatic reading of R. Kelly lyrics is just what you need