A 17-time Grand Slam title winner, Federer failed to serve out the first set when leading, 5-3, and was unable to put away the 19-year-old in the third despite saving two break points in his second service game.
The Swiss star gained a 5-4 lead but was unable dismiss Zverev, who used to two booming serves to close out the match and improve to 2-1 in their three career meetings.
Federer, 35, was playing in his second match since a six-month layoff because of knee and back injuries.
Federer's singles setback notwithstanding, Switzerland posted a 2-1 victory over Germany after the veteran star teamed with Belinda Bencic to record a 4-1, 4-2 win in the Fast Four mixed doubles match. Bencic also emerged triumphant in her singles match, posting a 6-3, 6-4 win over Andrea Petkovic.
--World No. 1 Andy Murray secured a spot in the Qatar Open quarterfinals with a 7-6 (6), 7-5 victory over Gerald Melzer in Doha, Qatar.
Murray, who is a two-time champion at the event, needed 78 minutes to win the first set and 2 hours and 23 minutes to put away the pesky Austrian.
"He played great, great tennis and was dominating large parts of the match," Murray said. "He made it extremely, extremely difficult. I was a bit lucky to through in two sets today."
Murray advanced to face Spaniard Nicolas Almagro, against whom he has won four of the five career meetings.
Defending champion Novak Djokovic posted a 6-3, 6-4 win over Horacio Zeballos and posed for a selfie with the Argentine after the match.
"I just have to mention making a selfie after the match was over, that was the first time that I ever had this kind of experience in my career," said Djokovic, a 12-time Grand Slam champion. "So, Horacio, well done. Very original."
Djokovic faces qualifier Radek Stepanek of the Czech Republic in the quarterfinals. The 38-year-old Stepanek, who notched a 6-3, 6-2 victory over wild-card entrant Arthur De Greef of Belgium, is the oldest ATP Tour quarterfinalist since then-42-year-old Jimmy Connors advanced to the final eight at the 1995 Halle tournament.
Third-seeded Tomas Berdych recorded 13 aces to topple fellow Czech Jiri Vesely 7-6 (5), 1-6, 6-1. Berdych faces fifth-seeded Frenchman Jo-Wilfried Tsonga, who cruised to a 6-1, 6-3 win over German Dustin Brown.
--Slovakian qualifier Jozek Kovalik recorded a 7-6 (5), 5-7, 7-5 victory over top-seeded Marin Cilic of Croatia in a second-round match at the Aircell Chennai Open in Chennai India.
Jovalik, who is ranked No. 117 in the world, overcame 18 aces and saved nine of 11 break points to put Cilic away in 2 hours and 48 minutes. He advances to face Russian Daniil Medvedev, who posted a 6-4, 6-3 triumph over eighth-seeded Yen-Hsun Lu of Taipei.
Also on Wednesday, third-seeded Spaniard Albert Ramos-Vinolas breezed to a 6-2, 6-0 win over Steve Darcis of Belgium. Vinolas will meet Israeel's Dudi Sela, who cruised to a 6-2, 6-2 victory over Korean Hyeon Chung.
--Stan Wawrinka of Switzerland continued his dominance of World No. 29 Viktor Troicki with a 7-6 (5), 6-4 triumph at the Brisbane International.
Wawrinka improved to 7-0 versus Troicki in their head-to-head series. He advanced to the quarterfinals to face Great Britain's Kyle Edmund, who breezed to a 6-3, 3-1 (retired) win over sixth-seeded Frenchman Lucas Pouille.
Also on Wednesday, Japan's Kei Nishikori posted a 4-6, 6-4, 6-3 win over American Jared Donaldson and Australian Jordan Thompson outlasted eighth-seeded Spaniard David Ferrer, 4-6, 7-5, 7-5.