Micky Rosenfeld, a police spokesman, said the victim was in his 40s and was not immediately identified after the shooting Friday.
The victim had shouted "Allahu akbar" (God is great) and may have taken an object out of his pocket before being shot, The Jerusalem Post said.
The security guard, who is employed by a private firm contracted to secure the plaza's entrances, fired several shots at the victim, The New York Times said.
Part of the plaza remained cordoned off hours after the shooting while police investigated the incident.
Tensions generally run high around the holy site, where Jewish worshipers regularly pray before the remnant of the retaining wall of the Temple Mount.
Muslims pray at Al Aksa Mosque, on a plateau above the wall.
Celebrity Families of 2014 [PHOTOS]