Gorzelanny, who had retired the Nationals in order in the first, was replaced by Alfredo Figaro.
Figaro worked three scoreless frames before Washington began chipping away with single runs in four of the last five frames.
Ian Desmond scored the game's first run on a fielding error by Milwaukee third baseman Jeff Bianchi in the fifth and Harper delivered what turned out to be the game-winning run in the sixth with his 16th homer.
Zimmermann (13-6) left after throwing exactly 100 pitches. He gave up four hits with four walks. Rafael Soriano fanned two in the ninth for his 27th save.
Figaro (1-3) was charged with the loss, giving up a run on two hits over four innings.
Duggar sisters unveil Christian dating rules in new book
LGBT community has 'bullied the American people': Bachmann