Gordon, who was elected to run the organization in June 2005, cited differences within the group, including with its board.
"We want it to be a social justice organization; he wanted it to be more of a social service organization," said Julian Bond, the chairman of the NAACP board.
Gordon, 61, joined the NAACP after more than three decades in the telecommunications industry. Prior to coming to the NAACP, he had been an executive for Verizon Communications.
"I believe that any organization that's going to be effective will only be effective if the board and the (chief executive officer) are aligned, and I don't think we are aligned," Gordon said. "This compromises the ability of the board to be as effective as it can be."
Many in the civil rights community expressed disappointment at Gordon's decision, the Times reported, particularly because his announcement came on the 42nd anniversary of Bloody Sunday, when police attacked civil rights protesters in Alabama. Bloody Sunday is known as a turning point in the battle for federal voting rights legislation.
NBC reportedly holds celebs hostage to Jimmy Fallon's show
Ray Liotta sues skin care company over use of likeness