Spitz won seven gold medals in the 1972 Olympics and Phelps had dreams of breaking or tying that record in Athens. But those hopes ended with a loss in the 400-meter freestyle relay Sunday night and the setback Monday evening.
Thorpe captured the 200-meter freestyle, the only individual event in which he will face Phelps.
Phelps had to settle for third when Pieter van den Hoogenband of the Netherlands edged him by .09 of a second for the silver.
The American swimming team bounced back from that disappointment by winning two gold medals Monday night in the men's and women's 100 backstroke.
Aaron Peirsol won the men's backstroke with Markus Rogan of Austria and Tomomi Morita of Japan claiming the silver and bronze.
China's Juo Xuejuan won the women's 100 breaststroke.