The victims, a man and a juvenile, were pulled out of the water near the Golden Gate Bridge around 5:45 p.m. PST Friday, the San Francisco Chronicle reported.
They were in the water for more than 30 minutes before rescuers pulled them onto a fireboat.
A third man was pulled out to sea by a wave, but managed to make it back to shore alive, said Matt Barnes, battalion chief for the Southern Marin Fire Department. The survivor was treated and released at the scene.
Barnes said a cause of death for the two fishermen had not been determined.
"The water is very cold out there," he said. "The air is very cold this time of year. Hypothermia could have played a factor."