Cast member Amber Riley performed "I Will Always Love You" and producers added a dedication card at the end of an episode last month, shortly after Houston died at age 48.
A show source told E! the new episode was planned after Houston's death and will "not be a typical tribute."
"It's using Whitney's music as a way to say goodbye," the source said. "A way to respectfully say thank you for her music."
"Glee" co-star Lea Michele mentioned the episode via Twitter Friday.
"Starting a new episode today. A very special tribute to one of the world's most amazing singers. Very excited about this one," Michele said.
An air date has not yet been set for the episode.