Stillman also notched an assist and Bryan Allen accounted for the other goal in the Panthers' third win in their last four outings.
Tomas Vokun ended the night with an impressive 47 saves for the Panthers. Antero Niitymaki started in goal for Tampa Bay but was pulled after allowing five goals on 18 shots. Rookie Dustin Tokarski finished the game with five stops.
Stillman scored twice in the first period to give the Panthers a 2-0 lead. Tampa came back to tie it 2-2, but Allen scored at 17:03 of the first period to make it 3-2.
Olesz added a pair of goals in the second period to close out the scoring.
Tampa Bay outshot Florida 21-5 in the final period but could not get past Vokun.