Some 900 people were evacuated at about 10:30 a.m. local time at the 1,000-foot skyscraper near London Bridge.
Seven fire engines and 50 firefighters responded to the scene, a spokesman for the London Fire Brigade said.
"We have got quite a few fire engines there because of the nature of the building," he said. "We are investigating reports of smoke, but there is no suggestion of a fire at this stage."
The building, the tallest in western Europe, houses offices, apartments, hotels and restaurants. It was opened in February 2013.