Graham said on NBC's "Meet the Press" that Afghanistan was one component of the struggle against al-Qaida and its ilk and that failure in Afghanistan would have regional consequences.
"They will kill every moderate who tried to help us, and no one in the future will step up," said Graham. "It will destabilize Pakistan beyond what exists today. It will be a colossal national security mistake."
Graham said he was concerned that the cost of the Afghanistan war was fueling an isolationist streak within the GOP that ignores the marathon nature of the conflict.
"By 2016, if we do this right, we can have troops in Afghanistan and Iraq, the same number as South Korea," Graham said. " If we accelerate withdrawal right now because we're war weary, we're going to lose this war."
Sen. Dick Durbin, D-Ill., told NBC that the death of Osama bin Laden meant the time was right for the United States to make a significant withdrawal from Afghanistan and turn more of the nation's security over to the Afghan government.