Plastic balls covered with Nazi symbols were discovered Sept. 25 and Oct. 11 in the backyard of the home in which Michael Brown and his wife Michelle live, The Port Huron Times Herald reported Wednesday.
The African-American couple has told authorities they believe neighbors are responsible.
St. Clair County Sheriff Tim Donnellon says deputies have spoken to both couples and there is no proof linking the golf balls to the people who rent the home next door to the Browns.
Donnellon said neighborhood disputes are not uncommon, but racially charged ones are not regularly seen.
"We take these things seriously," he said.
The neighbors say they think the Browns's accusations are responsible for their landlord's decision to evict them.