Horford in put 15 points to go with nine rebounds while Williams sank 6-for-12 shots coming off the bench as the Hawks notched their third win four contests.
Zaza Pachulia contributed eight points with 11 rebounds and Jeff Teague provided nine points and seven assists in Atlanta's victory.
Orlando, meanwhile, lost for the seventh time in eight outings despite 11 points and seven boards from Glen Davis.
Nikola Vucevic hauled down 13 rebounds in the setback.
2014: The Year in Fashion [PHOTOS]
LeBron James reveals his new Nike shoes