Two people are missing, The Indian Express reported.
"While four persons have been reported missing, two bodies burned beyond recognition are suspected to be that of [Mridul Kumar] Bhattacharyya and his wife, Rita," local law enforcement official Meenakshi Sundaram said.
"The manager of the tea garden and his wife were also in the bungalow when the mob set it on fire but their whereabouts are not known."
Workers surrounded the bungalow at Kunapathar in Assam state Wednesday and set the house and two vehicles on fire, which police said followed management asking some workers to leave their living quarters, the BBC reported.
Sundaram said two bodies were found in the debris.
Three people were detained, the BBC reported.
More than half of India's tea comes from estates in Assam.
Officials said Bhattacharyya faced protests two years ago at another tea estate he owned. In that dispute, workers set fire to a tea factory near Guwahati after Bhattacharyya allegedly fired into a crowd protesting a reported attack on a woman.