His appearance in Tripoli has raised concerns he may be building an al-Qaida network in the country, CNN reported.
It isn't clear if Libya is aware 48-year-old Abu Anas al-Libi is in the country or if Western governments have asked Libya to arrest him, the report said.
The United States and Libya do not have an extradition treaty.
The security situation is delicate in Libya and analysts suggested al-Libi may not have been arrested because of the large number of former jihadists there.
It isn't clear when al-Libi came to Libya. A Western intelligence agency took photos of him during last year's civil war. In December, Libyan authorities provided the United Nations with his street address in Tripoli.
It isn't known if al-Libi is still active in jihadist causes.
The United States put a $5 million bounty on al-Libi after he was indicted in 2001 for his involvement in al-Qaida attacks on U.S. embassies in Nairobi and Dar es Salaam on Aug. 7, 1998. More than 200 people died in the attacks.