The 60-year-old TV personality reportedly has demonstrated some erratic behavior and done two stints in rehab.
People magazine said no reason was offered for his departure from the show, however.
"Pat O'Brien is a talented broadcaster who has been a part of 'The Insider' family from the very beginning," a representative for the show said in a statement. "Although we have decided to part ways, everyone at CBS Television Distribution and 'The Insider' greatly appreciate and respect all his contributions to our show. We wish Pat much success in his future endeavors."