The patch, written in German, reads "My honor is loyalty" -- the former slogan of Hitler's SS during World War II. It is seen on the soldier's right arm, Radio France Internationale reported Saturday.
Military officials said they are looking for the soldier in the photo and will suspend him.
"This is an unacceptable attitude that doesn't reflect the reality of the armed forces," said army spokesman Col. Gilles Jaron.
The French army has about 1,600 soldiers deployed in the CAR as part of an international peacekeeping force.
NBC reportedly holds celebs hostage to Jimmy Fallon's show
Aaron Carter is still in love with Hilary Duff