The report suggests the Fed is in no hurry to offer further economic stimulus, analysts said.
It was the fifth straight loss for the Dow Jones industrial average, which fell 48.59 points, or 0.38 percent, to 12,604.53.
The Standard & Poor's 500 index was virtually unchanged, nudging just 0.02 points down to 1,341.45. The Nasdaq composite index fell 14.35 points, or 0.49 percent, to 2887.98.
The market was lower for most of the day and then fell sharply after the Fed minutes were released. Stronger energy stocks helped ease losses toward the end of the trading day.
There were 1,605 advances and 1,434 decliners on the New York Stock Exchange with nearly 768 million shares traded. Composite volume was close to 3.4 billion.
The 10-year U.S. treasury note was down 5/32 to yield 1.522 percent.
The euro was up at $1.2241. Against the yen, the dollar rose to 79.66.
In Tokyo, the Nikkei 225 index gave up 0.08 percent, 6.73 points, to 8,851.00. In London, the FTSE 100 was down 0.41, or 0.01 percent, to 5,664.48.
Swim Week Miami Beach 2014 [PHOTOS]