Guards spotted the intruder but were unable to leave their posts to give chase, The Sun reported Monday.
The guards radioed for help but were ignored, a source told The Sun.
"Security was a total shambles. The burglar climbed over the front gate, then got over another gate and found a metal box with the keys inside. The box is supposed to be kept locked at all times but it was open," the source said.
"It went from bad to worse. Guards saw him but could not leave their positions to chase him. They tried to radio a night watchman but got no response."
Officials said the crown jewels were never under threat of being stolen, but security procedures "were not carried out to the expected standard."
"A staff disciplinary procedure is under way," said a spokesman for the Historic Royal Palaces, which runs the tower, said.