Gyllenhaal made his entrance after Corden admitted to thinking about the actor before the pair launched into a version of Houston's "Greatest Love of All" titled "Greatest Gyllenhaal of All."
The duo sang about how great Gyllenhaal appearing on the show could have been and how the actor's sister Maggie is actually the superior sibling.
"Every night I watch The Late Late Show/ People need a host to look up to/ I really wish I could be on your balcony/ One day I hope to see my face on Carpool Karaoke," Gyllenhaal sang.
Gyllenhaal will next be seen as the villainous Mysterio in Spider-Man: Far From Home, which arrives in theaters on July 5.
Corden recently made headlines when he formed a U.K. dodgeball team consisting of Harry Styles, Benedict Cumberbatch, John Bradley and Late Late Show bandleader Reggie Watts to take on Michelle Obama's U.S. team featuring Mila Kunis, Melissa McCarthy, Kate Hudson, Allison Janney and Lena Waithe.