Adam Strange was swimming from Maori Bay into Muriwai approximately 650 feet offshore when the attack occurred, TVNZ reported.
Fisherman Pio Mosie said he heard Strange yell out "a shark!"
"We looked up and there was blood everywhere and the shark was attacking him," Mosie told TV ONE.
"Then the shark stopped and we yelled at him to swim to the rock, and he raised his hand up, but as soon as he raised his hand up the shark pulled him down. When he came up his head was down and now we know he was dead."
Police said they believe the shark was a great white, or perhaps more than one bronze whaler shark, though they are not sure.
Police said they had to shoot at the shark and others that gathered due to the blood in the water to get them to leave the area.
Muriwai Surf Lifesaving Chairman Tim Jago said shark attacks in New Zealand are rare. The last death from a shark attack was in 1975, TVNZ reported.
"This is one out of the hat for us I don't think anyone's seen anything this big at this beach ever," Jago said of the shark.
2014: The Year in Music [PHOTOS]