Retailer Migros said the creamer packs bearing the faces of the fascist leaders were packaged by Bern company Karo-Versand and were sent out to restaurants and cafes that purchase creamer wholesale from Migros.
Tristan Cerf, a spokesman for Migros, said internal controls had failed and the company plans to "sharpen" its controls to make sure similar incidents are avoided in the future.
Karo-Versand officials said the labels were part of a larger line designed to look like classic cigar labels.
Migros announced it would cut ties with Karo-Versand, which company director Peter Walchli said could close down the firm and spell the end of Switzerland's market for collectible creamer lids.
"We are the only company in Switzerland to keep a record of these cream pots. If we can no longer do it then 5,000 people will have to find another hobby because they will no longer know if a series is complete or not," he told newspaper 20 minutes.
Walchli said the series was intended only for collectors and was not meant to be sent out to regular customers.
However, Cerf said Karo-Versand's explanation would not change Migros' plans.
"Whether the pots were destined for collectors or for Migros, it's unacceptable. Hitler has no place on a cream pot lid!" Cerf said.