It was the second time in three tournaments that Federer lost a second-round match. He also dropped a straight-set decision to 114th-rated Federico Delbonis last week in Germany in a tournament semifinal.
Brands, who lost to Federer in three sets last week, won Thursday 6-3, 6-4. Federer was unable to finish off any of his five break chances and Brands posted a break in each set to advance to the quarterfinals.
It was a day of upsets at the Swiss Open with all three seeded players on the schedule losing.
Victor Hanescu beat No. 8-seeded Roberto Bautista Agut 7-6 (7-2), 3-2 when Bautista Agut retired.
Also Thursday, Marcel Granollers came out a 6-2, 7-5 winner over Delbonis.
Brands and Hanescu are paired against each other in Friday's quarterfinals with fourth-seeded Juan Monaco and No. 6-seeded Mikhail Youzhny also playing in that half of the draw. Granollers meets Haase with the winner moving to the semifinals against the winner of a match between second-seeded Stanislas Wawrinka and No. 5 Feliciano Lopez.