Brown was invited to his ex-wife's funeral despite tension with the late recording star's family, who reportedly blame him at least in part for Houston's years of drug and alcohol abuse.
But the Newark Star-Ledger reported Brown left the New Hope Baptist Church minutes before the service began because the nine people he brought with him were not allowed to sit with him in the family section of the church.
"They were asked to leave," a person with knowledge of the situation told the newspaper, referring to Brown's entourage.
Although Brown was not asked to go, the insider said, "He left with the other people."
The Star-Ledger said Brown appeared visibly upset as he climbed into a black Cadillac SUV and rode away.
Brown, also a recording artist, issued a statement to CNN after the funeral ended, explaining: "My children and I were invited to the funeral of my ex-wife Whitney Houston. We were seated by security and then subsequently asked to move on three separate occasions. I fail to understand why security treated my family this way and continue to ask us and no one else to move. Security then attempted to prevent me from seeing my daughter (with Houston) Bobbi Kristina. In light of the events, I gave a kiss to the casket of my ex-wife and departed as I refused to create a scene. My children are completely distraught over the events. This was a day to honor Whitney. I doubt Whitney would have wanted this to occur. I will continue to pay my respects to my ex-wife the best way I know how."
In addition to Bobbi Kristina, who was at the church with Houston's family, Brown has four other children, the youngest of whom is 2. He did not say how many of his kids accompanied him to Saturday's service.
Houston died last week at the Beverly Hilton Hotel in Los Angeles at the age of 48. Officials have said they are waiting to determine the cause of her death until after toxicology test results are analyzed. No foul play was suspected.
Swim Week Miami Beach 2014 [PHOTOS]