Taylor hauled down 10 rebounds and Semaj Christon added 13 points for the Musketeers (16-11, 8-5 A-10).
Xavier's victory was its second in three contests.
Memphis (24-4, 13-0 C-USA) got 14 points and seven boards from Geron Johnson, but it wasn't enough to prevent their first loss since Dec. 15.
Joe Jackson reached double figures with 11 points for Memphis.
Millions of Getty images now available for free via embed tool
NBC reportedly holds celebs hostage to Jimmy Fallon's show