The topic was brought up on Wednesdays "Ellen DeGeneres Show" -- the same show on which Garner first revealed her baby was going to be a girl.
DeGeneres asked the 34-year-old actor: "Will you have more? Should I ask that question?" People magazine reported.
"You know ... sure, I would love to," Affleck responded.
"Many? A lot?" DeGeneres asked.
Suggesting, however, that "some people" would want "five or six" kids and would "love to have a large family," Affleck told DeGeneres, "I think I should probably talk to my wife before I talk to you."
The topic of Affleck's recent hiatus from film work was brought up as well.
"I took about a couple of years off, definitely for a full year I didn't do anything and I did this (the period drama 'Hollywoodland'), and then the rest of the time I spent directing a movie," said Affleck, referring to the Boston detective drama "Gone, Baby, Gone," to be released next year.
Celebrity Families of 2014 [PHOTOS]