The Rookie of the Year finalist was walking out of an airport when he was asked if he would like to play with LeBron.
"If he comes, I'll play with him," Simmons said.
Simmons said "of course" when asked if he would like to play with James.
The four-time NBA MVP averaged 27.5 points, 9.1 assists and 8.6 rebounds per game this season, starting 82 games for the first time in his career. James, 33, is making $33.2 million this season and has a $35.6 million player option next season.
Cavaliers owner Dan Gilbert recently told ESPN that James' potential free agency this offseason is not a distraction.
"Like him, I think he's said this, we're just focused -- and it's the truth, it's not avoiding the question -- we're literally focused on [the now]," Gilbert said. "It's just so intense in the playoffs and getting to the Finals and now the Finals, I don't think -- and I know he doesn't -- I don't think even our guys are talking [about anything other than] what's in front of them."
"So, everybody knows, obviously what we want to happen and we'll take the Finals first and take it from there."
Several NBA cities have taken it upon themselves to recruit James this season, as have several Minor League Baseball teams. In February, a Philadelphia-based company put up three billboards along a Cleveland highway, saying "complete the process" and "Philly wants LeBron."
James later told reporters that he knew about the billboards and that it was flattering.
"You could say it's a distraction. It's not," James said. "It's not a distraction."
"It is actually very flattering that I'm sitting here at 33 and in my 15th year and teams or guys -- I don't want to say teams because that becomes tampering -- but people in their respective city want me to play for them. That's cool I think."
James and Simmons have the same agent, Rich Paul of Klutch Sports Group.