The actress performed Spears' "Oops!... I Did It Again" as Fitzgerald and Armstrong on Thursday, turning the song into a duet as she switched between impersonating the classic singers.
"I have the musical taste of like a 75-year-old hermit woman who's like wearing a dress made out of spider-webs," Gilpin joked to host Jimmy Fallon.
"This is like what I would do when I was 6-years-old alone and then stoned at 17," she continued before launching into her performance.
Gilpin portrays Debbie/Liberty Belle on Netflix's Glow, a comedy-drama inspired by the Gorgeous Ladies of Wrestling -- a real life female wrestling league and television show from the 1980s.
Glow Season 3 is set to premiere on Aug. 9.