The Thunder won for the first time since Westbrook went out for the remainder of the season with a knee injury.
They had dropped two games to the Rockets, but finally eliminated Houston in six games and set up a Western Conference semifinal encounter with Memphis.
Memphis also advanced Friday with its fourth straight victory over the Los Angeles Clippers.
Houston held an eight-point lead late in the third period, but by the time the quarter was over the Thunder had inched in front by a point.
Oklahoma City then went on a 9-2 run to open the final period and held on from there to take a step toward its second straight conference title.
Martin came off the bench to score 25 points, hitting 7-of-13 from the field and all eight of his free throws.
He gave a much-needed assist to Oklahoma City superstar Kevin Durant, who scored 27 points to go with eight rebounds.