Houston won its fourth straight to climb within one game of .500.
The Astros scored an unearned run in the first thanks to a throwing error by second baseman Elian Herrera and the blast by Martinez in the third put Houston in front by three.
Houston still had a 3-0 advantage heading into the eighth and the Astros used five pitchers during that inning to escape with the lead.
Starter Lucas Harrell started on the mound in the eighth, but he was soon gone as the Dodgers loaded the bases with one out. A bases-loaded walk by Jerry Hairston drove in the lone Los Angeles run and Houston then got out of the inning when Fernando Abad struck out Ivan De Jesus and Fernando Rodriguez got A.J. Ellis on a fly ball to center.
Harrell (4-3) allowed five hits, walked three and did not record a strikeout.
Clayton Kershaw (4-2) took the loss, giving up two earned runs on five hits with six strikeouts in seven innings.