The Santa Clara County Sheriff's Office said none of the canoeists were wearing life vests.
"We're not sure why it happened," Deputy Kurtis Stenderup told the San Jose Mercury News. "There was a lot of wind out there but we'll have to look into it."
The names of the victims were not released. One was a 35-year-old man who was pulled from the water but died shortly after at a hospital. The second was a 30-year-old man whose body was recovered a few hours later.
The survivors were a 16-year-old girl and a 25-year-old man who said they were related to the men who drowned, the newspaper said.
Swim Week Miami Beach 2014 [PHOTOS]