Cast member Amber Riley performed "I Will Always Love You" and producers added a dedication card at the end of an episode last month, shortly after Houston died at age 48.
A show source told E! the new episode was planned after Houston's death and will "not be a typical tribute."
"It's using Whitney's music as a way to say goodbye," the source said. "A way to respectfully say thank you for her music."
"Glee" co-star Lea Michele mentioned the episode via Twitter Friday.
"Starting a new episode today. A very special tribute to one of the world's most amazing singers. Very excited about this one," Michele said.
An air date has not yet been set for the episode.
Celebrity Families of 2014 [PHOTOS]