Djokovic won the first 43 matches he played in 2011 and captured three of the four grand slam tournaments, but he will not be part of the semifinals at the closing event of the year.
Janko Tipsarevic defeated Djokovic 3-6, 6-3, 6-3 in Group A action, the second loss Djokovic suffered in group play. Djokovic came into the tournament with only four losses all year.
Berdych rallied past David Ferrer in the other group match 3-6, 7-5, 6-1. It was Ferrer who had first downed Djokovic in this tournament.
Ferrer had already clinched a spot in the final four and will meet Roger Federer in the semifinals. Berdych will face Jo-Wilfried Tsonga.
Federer, who went unbeaten in Group B, is trying for a record sixth championship. World No. 2 Rafael Nadal failed to reach the semifinals from Group B.