The Astros, who had lost three in a row, ended Tampa Bay's eight-game winning streak.
Cosart was called up from Class AAA Oklahoma City to make the start and the Astros had planned to send him back to the minors after one appearance.
That plan could be changed, however, after Cosart (1-0) pitched into the ninth inning.
He did not give up a hit until Ben Zobrist singled with one out in the seventh and Zobrist was immediately erased in a double play.
After issuing a leadoff walk in the ninth, his third of the game, Costart was replaced by Jose Veras. Tampa Bay eventually scored an unearned run in the inning, but Veras struck out Evan Longoria with runners on first and third to end the game. It was the 18th save for Veras.
Costart has been in the minors for four full seasons, having originally been drafted in the 38th round by Philadelphia.
He was trying to become the second pitcher to throw a no-hitter in his big league debut. The only one to have done so was Cincinnati's Bumpus Jones more than a century ago.
Both the Houston scores came in the first inning on a two-run single by Carlos Corporan off David Price (3-5).
Price went the distance, giving up eight hits.