Ballmer, 57, followed Microsoft co-founder Bill Gates as CEO in 2000. He will stay on to help in the search for his successor, alongside a special committee of the board of directors, including Gates, chairman of the company.
"There is never a perfect time for this type of transition, but now is the right time," Ballmer said in a statement.
Shares of Microsoft jumped 7 percent in early trading after the announcement to $34.84 a share.
“My original thoughts on timing would have had my retirement happen in the middle of our company’s transformation to a devices and services company,” Ballmer said. “We need a C.E.O. who will be here longer term for this new direction.”