He was named the school's new coach on Friday.
Jones spent the last six seasons as an assistant at Villanova. During his tenure there, the Wildcats compiled a 105-83 record, advancing to the postseason five times.
At Columbia, Jones is replacing Armond Hill, who was fired on March 10 after the Lions completed a 2-25 season. Jones becomes the 20th head coach in the 103-year history of the program.
"Joseph will bring a new level of intensity and excitement to Columbia men's basketball," said Director of Athletics John Reeves. "His dynamic personality and attention to detail will captivate players."
Jones, 37, has a reputation for being a strong recruiter. He was an assistant to Villanova Coach Jay Wright at Hofstra, and helped Wright and Villanova land one of the nation's best recruiting classes in 2002.
"Joseph's skill as a recruiter is his biggest asset," said Reeves. "He has demonstrated ability to recruit, and knows the New York area. He has been a key cog in the success at both Villanova and Hofstra."
Jones, 37, is the brother of Yale Coach James Jones.