The exchange between the Hall of Famer and the artist was recorded by cameraman Larry Schwartzwald, The New York Post reported Monday.
The newspaper says when Schwartzwald asked Jackson why he made the remark, the baseball player asked whether he, too, was Jewish.
"You're going to get me in trouble," Jackson told the artist, Peter Zonis.
Zonis, 49, insisted that Jackson is not anti-Semitic.
The two eventually settled on a price of $1,500 for the painting Jackson wanted.
"He (Jackson) was being nice and he knew I wasn't going to let him leave the scene" without buying the painting, Zonis told The Post.
A spokesman for the Yankees declined to discuss the incident, which took place Saturday afternoon.