The incident happened early Sept. 4 when university student Muhammad Ahmad Munir, 32, asked the female driver for the time, The (Montreal) Gazette reported.
Munir said the driver answered in French and he told her he didn't understand and asked for the time again.
He claims the driver then said "I don't speak English," and, in French, ordered him and about 20 other passengers off the bus. The driver also hit the vehicle's panic button, which automatically summons police to the scene, the newspaper said.
Societe de Transport de Montreal spokeswoman Isabelle Tremblay told the Gazette an internal investigation was still under way and she couldn't comment about the incident.