O'Brien is to drop by the show May 17.
He succeeded Letterman as host of NBC's "Late Night" in 1993 after Letterman left to begin "Late Show" on CBS. O'Brien served as host of "Late Night" until 2009, and then became emcee of "The Tonight Show" until January 2010.
In November 2010, he began his late-night cable talk show, "Conan," on TBS.
The visit will be O'Brien's fifth to "Late Show." He last sat down for an interview with Letterman Jan. 13, 1999.
Musician John Mayer will sit in with Paul Shaffer and the CBS Orchestra on the May 17 episode.