Koskinen has served as non-executive chairman of Freddie Mac from 2008-11 and acting chief executive officer in 2009, and was deputy mayor and city administrator of Washington from 2000-03.
He would succeed acting Commissioner Danny Werfel, appointed in May after it was revealed that the agency targeted political organizations seeking tax-exempt status for further scrutiny.
The revelation led to the resignation of Stephen Miller as acting commissioner.
"John is an expert at turning around institutions in need of reform. With decades of experience, in both the private and public sectors, John knows how to lead in difficult times, whether that means ensuring new management or implementing new checks and balances," Obama said in a statement. "Every part of our government must operate with absolute integrity and that is especially true for the IRS. I am confident that John will do whatever it takes to restore the public's trust in the agency."
Treasury Secretary Jack Lew called Koskinen the "right person" to tackle the position at this "important time."
"Because John has a clear understanding of how to make organizations more effective and an unshakable commitment to public service, he will be an exceptional leader who will strengthen the institution and restore confidence in the IRS," Lew said in a statement.