Nkomo had been ill with cancer for several years. The Zimbabwean newspaper NewsDay reported he was rushed to St. Anne's Hospital after his health suddenly worsened.
Mugabe appointed Nkomo vice president in 2009.
Nkomo participated in the struggle for independence in the British colony then-known as Rhodesia and against the white-ruled republic established by Ian Smith. He joined the Zimbabwe African People's Union in 1961 and spent two years in detention in the 1960s.
After the end of white rule in 1980, Nkomo served in Parliament and held a number of government offices. He continued to do so after ZAPU merged with Mugabe's Zimbabwe African National Union to form ZANU-PF in 1987.
In 2006, Nkomo, then speaker of Parliament, announced plans to seek the ZANU-PF nomination for the presidency. He withdrew when Mugabe said he would run again.
Nkomo was banned from travel to the United States as part of sanctions against the Mugabe government.