He replaces Steve Addazio, who took the head coaching job at Boston College this month.
Rhule, who has 14 years of college coaching experience, was previously on the Owls' staff from 2005-2011.
He spent last year on the staff of the NFL's New York Giants.
"Matt is the perfect person to take the Temple football program to the next level," Temple Director of Athletics Bill Bradshaw said. "He is an outstanding recruiter and in his six years at Temple was responsible for bringing many of the players that helped the program win its first bowl in 32 years."