After LSU had erased a 17-point, second-half deficit to tie the score 24-24, Mississippi's Bo Wallace marched the Rebels from their own 15-yard line, keeping the drive alive with a 14-yard pass to Ja-Mes Logan on third-and-11.
Wallace later scrambled for 10 yards and I'Tavius Mathers added a 12-yard run to the LSU 30, setting up Ritter's decisive kick.
Ole Miss (4-3, 2-3 SEC) broke a three-game losing streak.
Zach Mettenberger completed 19-of-33 passes for 274 yards and a touchdown but was picked off three times for the Tigers (6-2, 3-2).
2014: The Year in Music [PHOTOS]