The Thunder won for the first time since Westbrook went out for the remainder of the season with a knee injury.
They had dropped two games to the Rockets, but finally eliminated Houston in six games and set up a Western Conference semifinal encounter with Memphis.
Memphis also advanced Friday with its fourth straight victory over the Los Angeles Clippers.
Houston held an eight-point lead late in the third period, but by the time the quarter was over the Thunder had inched in front by a point.
Oklahoma City then went on a 9-2 run to open the final period and held on from there to take a step toward its second straight conference title.
Martin came off the bench to score 25 points, hitting 7-of-13 from the field and all eight of his free throws.
He gave a much-needed assist to Oklahoma City superstar Kevin Durant, who scored 27 points to go with eight rebounds.
Beyonce flaunts bikini body, Blue Ivy in vacation pics
Couple mistakenly served bag of cash at McDonald's drive-thru