The cannon was found Friday afternoon in the park's Ramble shed near the 79th Street transverse, WCBS-TV, New York, reported.
Along with the cannon were more than 800 grams of black powder still capable of firing and a cannonball.
Preservation workers for the Central Park Conservancy said the cannon was donated to the park around the time of the Civil War, and was on display in the park from the 1860s until 1996 when it was put into storage to protect it from vandalism.
The donation came from someone who salvaged the artillery from a sunken British frigate in the East River.
Tourists in the park Friday afternoon said they were shocked by the find.
"Something like that, it's surprising to be overlooked," Denise Night said.
"It seems like some people are pretty incompetent not to notice after all these years," agreed Steve Night.
The New York City Police Department safely removed the cannon, gun powder and cannonball from the park.
"We silenced British cannon fire in 1776 and we don't want to hear it again in Central Park," the NYPD said in a statement.
N.J. man wakes up from 10-hour sleep with knife in back
Campus cop fatally shoots Texas student during traffic stop