He is the only former prime minister for nearly 200 years not to be represented by a painting in the gallery.
Archives show two of Britain's leading portrait artists refused to paint him because they thought he would be too "difficult" a person, the Sunday Times of London reported.
Heath refused to sit for lesser-known artists whose names were put forward as willing to paint his portrait, the newspaper said.
The first artist to refuse the commission was Graham Sutherland in 1976.
"He wanted to be done by Sutherland and by nobody else," recorded a note by John Hayes, who was the gallery's director. Sutherland rejected the commission, believing it would be too awkward, the newspaper said.
In 1994, Heath decided he wanted Lucian Freud, now considered to be Britain's greatest living painter, to paint his portrait. But Freud, who produced a rather unflattering portrait of the Queen in 2001, refused to paint Heath. He, too, apparently thought the former prime minister would be too difficult a commission, the newspaper said.