Comey told ABC News, in a highly anticipated interview, while he doesn't believe everything he hears about Trump, he does believe he should not be president of the United States.
"I don't buy this stuff about him being mentally incompetent or early stages of dementia. He strikes me as a person of above average intelligence who's tracking conversations and knows what's going on," Comey said. "I don't think he's medically unfit to be president. I think he's morally unfit to be president."
Comey also revealed his belief that Hillary Clinton, the 2016 Democratic presidential nominee, would win the election factored into the investigation into her private email server.
This included his decision to tell Congress shortly before the election that the FBI was reviewing new emails regarding the email probe.
"It must have been. I don't remember consciously thinking about that, but it must have been because I was operating in a world where Hillary Clinton was going to beat Donald Trump, and so I'm sure that it was a factor," Comey said during the interview.
Trump said Monday Comey and others in the bureau had violated the law.
"[He] drafted the Crooked Hillary exoneration long before he talked to her, then based his decisions on her poll numbers. Disgruntled, he, McCabe, and the others, committed many crimes!" Trump said in a tweet.
Trump had previously criticized Comey after excerpts of his book and the interview were released.
"Unbelievably, James Comey states that Polls, where Crooked Hillary was leading, were a factor in the handling (stupidly) of the Clinton Email probe," Trump tweeted earlier Sunday. "In other words, he was making decisions based on the fact that he thought she was going to win, and he wanted a job. Slimeball!"
Excerpts of Comey's memoir, A Higher Loyalty, in which he describes his brief time as FBI director under the Trump administration, were issued last week. It will be released Tuesday.
In the book, Comey said the president asked him to be loyal, writing: "This president is unethical, and untethered to truth and institutional values. His leadership is transactional, ego driven and about personal loyalty."
"I never asked Comey for Personal Loyalty," Trump answered via Twitter. "I hardly even knew this guy. Just another of his many lies. His 'memos' are self serving and FAKE!"
Trump also tweeted: "The big questions in Comey's badly reviewed book aren't answered like, how come he gave up Classified Information (jail), why did he lie to Congress (jail), why did the DNC refuse to give Server to the FBI (why didn't they TAKE it), why the phony memos, McCabe's $700,000 & more?"
Friday, Trump described Comey as a "LEAKER & LIAR" who "should be prosecuted." Sunday, he intensified the rhetoric, writing, "Slippery James Comey, a man who always ends up badly and out of whack (he is not smart!), will go down as the WORST FBI Director in history, by far!"
Comey also mentioned Obama-era Attorney General Lorretta Lynch's "tortured half-out, half-in approach" to the Clinton email investigation and considered calling for the appointment of a special prosecutor. Former President Bill Clinton met with Lynch while their respective planes were parked on the tarmac in Phoenix in 2016, where she said they "talked about innocuous things."
Trump said, "Comey throws AG Lynch 'under the bus!' Why can't we all find out what happened on the tarmac in the back of the plane with Wild Bill and Lynch? Was she promised a Supreme Court seat, or AG, in order to lay off Hillary. No golf and grandkids talk (give us all a break)!"
Trump also blasted raids by the FBI of personal attorney Michael Cohen's office, hotel room and home last week. Federal prosecutors said Friday they focused on Cohen's "criminal conduct that largely centers on his personal business dealings" and not on communications between Cohen and Trump.
"Attorney Client privilege is now a thing of the past," Trump tweeted after the raids. "I have many (too many!) lawyers and they are probably wondering when their offices, and even homes, are going to be raided with everything, including their phones and computers, taken. All lawyers are deflated and concerned!"