The index that assigns 1985 a base value of 100 fell from 68.7 to 64.9, the Conference Board said.
The monthly Consumer Confidence Index is based on a survey of 5,000 households.
In May, consumers were less confident about current business conditions, but "more upbeat," about the prospect of rising income, said Lynn Franco, director of the Conference Board Consumer Research Center.
She said the numbers indicate the pace of economic growth "may moderate" in the months ahead.
In May, the number of respondents to the survey indicating economic conditions were "good" fell from 15.5 percent to 13.6 percent. The percentage of respondents indicating conditions were "bad" rose, climbing from 33.2 percent to 34.3 percent.
On the employment front, the percentage of consumers indicating jobs were "plentiful" fell from 8.4 percent to 7.9 percent, while the percentage indicating jobs were hard to find rose from 38.1 percent to 41 percent.