The monthly Consumer Confidence Index, which uses 1985 as a base year with an assigned value of 100, fell from 66 in April to 60.8 in May.
The Expectations Index, measuring consumer confidence in the economy six months down the road, also fell, dropping from 83.2 a month ago to 75.2.
In April, the monthly survey of 5,000 households found the number of respondents indicating economic conditions were "good" fell from 15.5 percent to 14.6 percent. The number of respondents indicating conditions were "bad" rose from 35.9 percent from 37.1 percent.
Consumers were less optimistic in May about the labor market than they were a month ago. The percentage of respondents indicating jobs were "plentiful" rose from 5.1 percent to 5.6 percent, while the number indicating jobs were "hard to get" rose from 42.4 percent to 43.9 percent.
"A more pessimistic outlook is the primary reason for this month's decline in consumer confidence. Consumers are considerably more apprehensive about future business and labor market conditions as well as their income prospects," said Lynn Franco, director of the Conference Board Consumer Research Center.
2014: The Year in Music [PHOTOS]