The Post reported Baldwin threatened a female reporter and referred to a black photographer as a "coon," "crackhead" and "drug dealer" when they approached him outside his Manhattan home during the weekend and asked for a comment about his pregnant yoga teacher wife, Hilaria, being sued by a former student.
However, TMZ said no racial epithets were heard on the audio it reviewed.
The celebrity news website also said Baldwin can be heard on the audio saying to the reporter, "I want you to choke to death," not, "I want to choke you to death," as the Post team alleged.
The Post said police were called and Baldwin and the photographer filed harassment claims against each other. However, the newspaper said the matter isn't expected to go any further unless one of the parties files a civil lawsuit.
Baldwin maintains he did nothing wrong.
"The claim of racist remarks is one of the most outrageous things I've heard in my life," the actor said in a statement to The Hollywood Reporter Monday.
Baldwin won numerous awards for his work on the TV sitcom "30 Rock," which recently wrapped its seventh and final season but he is also known for having a hot temper.
Several years ago, audio of him screaming at and calling his 11-year-old daughter "a rude, thoughtless little pig" was leaked on the Internet.
In 2011, the actor left his seat on an airplane and stormed into the bathroom when a flight attendant told him to put away his electronic device so they could takeoff. He apologized for the tantrum and said it was because he wasn't finished playing a session of the computer game "Words with Friends."
Baldwin, who identifies himself as a political liberal, is frequently critical of Rupert Murdoch's media empire, of which the Post is part, and the newspaper, in turn, for years called him "The Bloviator."
2014: Pregnant Celebrities [PHOTOS]
'True Detective' adds Kelly Reilly