Tanaka, 25, was considered one of the biggest possible acquisitions of the off-season and was considering a series of U.S. teams before settling on the Yankees.
The contract, through the 2020 season, but with an opt-out after the 2017 season, is reportedly worth $155 million.
"I felt we needed another starter. We were not where we needed to be, in my opinion," Yankees managing general partner Hal Steinbrenner told the New York Post.
"So this should not be a surprise because [Tanaka] was the best free-agent pitcher available. He is one of the greatest players Japan has ever produced. He is tough. He has thrived under pressure. He will fit in well to New York."
Tanaka was 24-0 with a 1.27 earned run average last season with the Tohoku Rakuten Golden Eagles in Japan's Pacific League. Over his seven-year career with the Golden Eagles he was 99-35 with a 2.30 ERA.
He won the Sawamura Award as Nippon Professional Baseball's best pitcher in 2011 and 2013.