Balloons, candles and other mementos had been placed along the side of a building in the neighborhood where Lawrence Campbell, 27, lived. They first appeared Sunday, hours after Campbell and Officer Melvin Santiago, 23, died.
A shrine to Santiago remained outside the Walgreen's drug store where the shooting took place.
"I had it taken down last night," Fulop said. "I am not going to let a few residents pretend like they express the views of a great city like Jersey City."
Investigators say Campbell shot Santiago at about 4 a.m. Sunday as the officer responded to a reported robbery at the Walgreen's. Witnesses say Campbell attacked a security guard and then told a shopper he was going to be "famous" just before the shooting.
While they were reluctant to give their names, some area residents said they believed the memorial to Campbell should be allowed to remain.
"There's two lives lost," one man at the Columbia Tavern told the Jersey Journal.
The Gun Violence Survivors Foundation announced plans Tuesday to raise $20,000 for a permanent memorial to Santiago.
"When I saw that the perpetrator of the heinous crime did it to seek fame, I thought the best thing we can do would be to work with Jersey City in building a lasting memorial in his honor, so that it will be Officer Melvin Santiago's name and legacy is remembered," said Christopher Hayes, the group's founder.