The official said the four were believed to have been held by the interior ministry, CNN reported Saturday.
It wasn't known why they were in custody.
The military personnel were in Libya "augmenting security at the U.S. embassy in Tripoli," the official said. They were taken into custody near the city of Sabratha, about 50 miles west of Tripoli.
The United States last month announced it was stepping up support for Libyan security forces, which have been been the objects of militia violence since the overthrow of Moammar Gadhafi.
A U.S. special forces team captured al-Qaida operative Abu Anas al Libi in October in a raid in Tripoli. The interim Libyan government called the capture a kidnapping.
Libi was wanted by the United States for his alleged role in the bombings of two U.S. embassies in Africa in 1998.