The Panthers have already ruled him out for Sunday's game against the New York Giants, but Olsen said there is a chance he could play the following week.
"I don't think it's out of the question," Olsen told ESPN.com. "If everyone, the doctors and I, feel like I'm ready I'm going to play. If I can play, I'm always going to err on the side of playing. But I want to be able to play and help the team."
Olsen missed nine games last season with a similar injury to the same foot. However, the screw that was placed in his foot to help the injury heal has prevented this break from being as problematic.
He also believes that dealing with last year's injury will help him be better prepared to contribute right away when he does come back this season.
Olsen, 33, had two catches for 33 yards in the opener before the injury. Last season, he had 17 receptions for 191 yards in just seven games.
He was named to the Pro Bowl each of the three years prior to the 2017 season.