The franchise previously hired Gaine in January 2018 after former general manager Rick Smith took a leave of absence to take care of his wife as she battled breast cancer. Gaine was signed to a five-year contract.
"After a thorough evaluation of our football operations, we have decided to relieve Brian Gaine of his duties as general manager," Texans CEO Cal McNair said in a statement. "Brian is a man of high character and we appreciate his contributions to our organization. We will begin an immediate search for a new GM.
"While the timing may be unusual, this decision was made in the best interest of our organization in our quest to build a championship team for the City of Houston."
Houston finished with an 11-5 record and won the AFC South last season. The Texans lost in the wild-card round to the Indianapolis Colts.
Chris Olsen, the organization's senior vice president of football administration, will lead the Texans' football operations until a replacement is found, according to the team.
Gaine has spent time with the New York Jets, Dallas Cowboys, Miami Dolphins and Buffalo Bills in his NFL executive career. He spent the 2017 season as the Bills' vice president of player personnel before joining the Texans.