Serbian media reported Milinkovic, 52, jumped from a multiple-story parking garage Tuesday in front of Zoran Vujic, an aide to Foreign Minister Ivan Mrkic who handles security and intelligence, ANSA reported.
Authorities said investigators had not determined what prompted the suicide.
Prosecutors in Brussels confirmed the NATO diplomat committed suicide, ANSA said.
"There are elements that clearly indicate it was a suicide and therefore the investigation can be considered closed," a representative for the prosecutor's office said.
A respected diplomat who had served in Vienna and Brussels, Milinkovic is survived by a wife and son.
Milinkovic was appointed ambassador to NATO Aug. 3, 2009. His previous assignments included ambassador to the Organization for Security and Cooperation in Europe for Serbia and Montenegro, ANSA said.
Milinkovic's death occurred on the second day NATO foreign ministers met at the alliance's headquarters in Brussels.
"The secretary general is deeply saddened by the news of the death of Ambassador Milinkovic," NATO Secretary General Anders Fogh Rasmussen said. "The ambassador was highly respected in his country and will be missed at NATO."