Martin, the two-time world time trial champion, was the 65th of 182 riders to leave Avranches for the 20.5-mile ride to Mont-Saint Michel in northwestern France. He set intermediate times that no rider challenged until Froome.
Froome, as the Tour's overall leader, was last off Wednesday but bested Martin's pace-setting time by 1 second at the first checkpoint and 2 seconds at the second.
He couldn't keep the pace over the final third of the race, ending up 12 seconds off Martin's winning time of 36 minutes, 29 seconds. Thomas De Gendt was third, 1 minute, 1 second back of Martin, with Richie Porte fourth, another 20 seconds back, and Michel Kwiatkowski fifth, 1:31 back of Martin.
Froome increased his overall lead from 1:25 after Tuesday's race to 3:25 ahead of Alejandro Valverde. Bauke Mollema moves into third place, 3:37 behind Froome.
Alberto Contador is 3:54 off the pace in fourth and fifth place belongs to Roman Kreuziger, who is 3:57 back of Froome.
The riders head from Fougeres southeast to Tours in central France, a relatively flat route covering 135.5 miles, for Stage 12 Thursday.