Federer, seeking a gold medal for Switzerland, downed Denis Istomin of Uzbekistan 7-5, 6-3 a month after winning the Wimbledon title.
His quarterfinal opponent will be American John Isner, who got past Janko Tipsarevic 7-5, 7-6 (16-14).
Azarenka had to work to survive against Nadia Petrova of Russia 7-6 (8-6), 6-4. She will now meet Angelique Kerber of Germany, who won two tiebreaks in defeating Williams 7-6 (7-5), 7-6 (7-5).
On a day of British success around other Olympic venues, Andy Murray added to it with his 4-6, 6-1, 6-4 victory over Marcos Baghdatis of Cyprus. Nicolas Almagro of Spain will await Murray in the quarterfinals after knocking off Steve Darcis of Belgium 7-5, 6-3.
One day after surviving a 3-hour third set in his win over Canadian Milos Raonic, Jo-Wilfried Tsonga of France knocked off Feliciano Lopez of Spain 7-6 (7-5), 6-4.
Tsonga will take on Novak Djokovic of Serbia in the quarterfinals after the second-seeded player rallied to dismiss Lleyton Hewitt of Australia 4-6, 7-5, 6-1.
In the women's draw, Serena Williams routed Vera Zvonareva of Russia 6-1, 6-0. Isner and Serena Williams are the only Americans left in the singles portion of the tournament.