Jackson was among three Memphis players who made 1-of-2 free throws down the stretch and it proved just enough to survive a last-minute charge by the Gaels.
The Tigers (31-6) will meet third-seeded Michigan State in a Midwest Region third-round game Saturday. St. Mary's, which won a First Four game Tuesday, ends its season 28-7.
St. Mary's never led but got within 54-52 when Eividas Petrulis banked in a three-pointer from the right side with 5 seconds to go. The Gaels got a chance to win when Memphis was unable to get the ball in cleanly with 1.9 seconds left but a desperation shot was off.
Memphis was also unable to put the game away with free throws. Adonis Thomas, Jackson and D.J. Stephens all were 1-of-2 in the game's final 38 seconds to just stem the St. Mary's comeback.
Jackson ended up with a team-high 14 points while Thomas and Tarik Black each had 12. Memphis was 9-of-18 from the free-throw line for the game.
St. Mary's Brad Waldow, on 7-of-11 shooting, led all scorers with 17 points.