The City Council's vote on the motion, which came after Ford admitted having bought illegal drugs in the past two years after last week having said he had smoked crack cocaine, was 37-5, with Ford and his brother among those on the losing end, the National Post reported.
"I effed up," the embattled mayor said with a shrug of his shoulders just prior to the vote.
The newspaper said council members later broke out laughing when Ford tried to introduce a motion calling for all them to undergo drug and alcohol testing.
The Toronto Star reported a poll found 75 percent of city residents think Ford should resign or take a leave to seek treatment.
At the heated meeting, Ford denied threatening Councilman Denzil Minnan-Wong, who sponsored the motion for a leave of absence. Minnan-Wong said Ford stood close to him in "a threatening way."
Ford said he was just asking Minnan-Wong to sit down.
"I did not threaten him in any shape or form," the mayor said. "He knows that. I asked him to take his seat. I have nothing to apologize for."
Ford has been under fire since the Toronto Star reported he had been videotaped smoking crack. When Minnan-Wong asked him if he had bought illegal drugs in the past two years, Ford paused and responded "Yes, I have."
"Our city's reputation has been damaged," Councilwoman Jaye Robinson said, reading the letter. "Together we stand to ask you to step aside and take a leave of absence. Let's get on with city business."
Ford, who admitted publicly last week that he had smoked crack while in a "drunken stupor" has refused to step down and says he plans to run again when his current term ends in 18 months. He has predicted the election will be a "bloodbath."
The Ipsos Reid poll was conducted from Friday to Tuesday on behalf of CTV News/CP24/CFRB Newstalk 1010 Radio. It surveyed 665 Toronto residents online. The margin of error is 4.3 percentage points.
2014: The Year in Music [PHOTOS]