The 35-year-old reality star discussed his relationship with Caitlyn during Monday's episode of The Hills: New Beginnings.
"I didn't really grow up with my dad," Brody said. "When I was really little we got to spend a little time. When he started, obviously, the family with the Kardashian family, I didn't really see much of him."
Brody is the younger son of Caitlyn and her second wife, Linda Thompson. Caitlyn is also parent to Brandon Jenner with Thompson, Burt Jenner and Casey Marino with Chrystie Scott, and Kendall Jenner and Kylie Jenner with Kris Jenner. Caitlyn was stepparent to Kris' children Kourtney, Kim, Khloe and Rob Kardashian during their marriage.
Brody elaborated on his relationship with Caitlyn in a confessional shot during the episode. Caitlyn, formerly known as Bruce Jenner, came out as transgender in April 2015.
"Bruce was just never really around -- every so often maybe once every couple years. Wasn't around for my graduation, wasn't around for most of my birthdays so, yeah, it really wasn't any relationship," Brody said.
"Eventually, we started to become closer but then all of a sudden, it's like, I was just getting to know Bruce and then Bruce became Caitlyn," he added. "She's totally different. It's awesome, it's great. She's happy, she's running around. She's off living her life, so what I've learned is to just not expect too much from her."
Brody previously told E! News in 2015 his relationship with Caitlyn had improved since her coming out.
"Caitlyn I get along with a little more, because Caitlyn is actually who she really is and she's living her life finally now as the person that she always was. Caitlyn is a much freer, happier person and I think that's very special," he said.