The quake struck 15 miles southeast of the town of Buli in central Taiwan at a depth of about 8.7 miles, the U.S. Geological Survey reported.
Taiwan's official Central News Agency said a male tourist, Liu Chin-lu, was killed by falling rocks in Alishan, Chiayi County, during the temblor.
A second man, identified only as Hsiao, was also killed by falling rocks while he was visiting a mountainous area in his agricultural field to inspect a water supply, the news agency said.
A mudslide near a waterfall in the township of Jhushan in Nantou County buried an angler, who was still missing Sunday evening. A helicopter airlifted a second angler to a nearby hospital, the Central News Agency reported.
A woman driving in Chiayi County and another person in Jhushan in Nantou County were also hit by falling rocks and suffered critical injuries, the news agency reported.
A road collapse in Jhushan hurt one man, who sustained an injury to the head and multiple fractures.
In Nantou, Chiayi and Yunlin Counties, 79 other less serious injuries were reported, the news agency said.
A tremor that had a magnitude of 5 was felt throughout Taiwan after the initial quake, the National Fire Agency said.