Ja left his seat before Trump said North Korea presents a clear danger and its weapons program could deliver an "unthinkable loss of human life," leaving his North Korean subordinate to stay behind to listen to the speech, CNBC reported.
In the course of the speech, Trump said if the United States is "forced to defend itself and its allies, we will have no choice but to totally destroy North Korea."
Pyongyang's diplomats at the U.N. later told NBC they "boycotted" the speech.
Ja was seated in the front row of the assembly hall, facing the podium where Trump would deliver his speech.
The North Korean ambassador was assigned to sit in the front row through a random seating assignment at the U.N., South Korean news service News 1 reported.
The North Korean delegation may have planned the boycott ahead of time, after a White House official said Trump would devote at least a part of his speech addressing North Korean provocations.
"No nation on Earth has an interest in seeing this band of criminals arm itself with nuclear weapons and missiles," Trump said. "Rocket Man is on a suicide mission for himself and for his regime."
Earlier over the weekend, Trump had used the term to refer to North Korean leader Kim Jong Un in a tweet.