The newspaper said Brown had a heated argument with his bodyguard, identified only as Big Pat, while their private plane stopped in Bermuda to refuel on its way back to the United States from West Africa where Brown performed.
It was unclear whether the disagreement became physical.
"I'm not sure of the details of what happened on the plane but his plane came in, got serviced and everything was finishing and they were getting ready to leave," the airport worker told the Sun.
"Then there was a stall. The security guard got off first. He was standing outside talking to the pilot. Then Chris Brown and another guy came off," the source said. "They had to come inside to Customs. They then went outside to have a cigarette. The security guard came inside [an airport building] and said he wasn't getting back on the plane. Chris went back through Customs and the plane left at about 2:20 p.m."
The newspaper said the bodyguard took a JetBlue plane home.
Brown, 23, has seen his popularity diminish after he pleaded guilty to assaulting his girlfriend, singer Rihanna, in 2009. He has since gotten into other high-profile altercations with artists, photographers and fans, as well.