Taylor hauled down 10 rebounds and Semaj Christon added 13 points for the Musketeers (16-11, 8-5 A-10).
Xavier's victory was its second in three contests.
Memphis (24-4, 13-0 C-USA) got 14 points and seven boards from Geron Johnson, but it wasn't enough to prevent their first loss since Dec. 15.
Joe Jackson reached double figures with 11 points for Memphis.
Duggar sisters unveil Christian dating rules in new book
Ohio bar shooting arrested, charged with murder