Kostunica had been trying to convince his Cabinet members to reject increased ties with the international group, the BBC reported Saturday.
"The government of Serbia has no united policy any more on an important issue related to the future of the country -- Kosovo as a part of Serbia," Kostunica said.
"Such a government could not function any more."
The moderate nationalist's resignation came after Serbian President Boris Tadic said the only way Serbia could effectively oppose the move would be to join the European Union.
Serbia has openly opposed Kosovo's decision to declare its independence in February and has been supported in its opposition by Russia, the BBC said.
Yet most European Union members, along with the United States, have officially acknowledged the region's independence from Serbia.