Bartending robot interprets body language of thirsty customers

Bartending robot James is programmed using body language study of real bar patrons and bartenders.

The bartending robot James, named after the EU project 'James' (Joint Action in Multimodal Embodied Systems). (Credit: fortiss GmbH/University of Bielefeld)
The bartending robot James, named after the EU project 'James' (Joint Action in Multimodal Embodied Systems). (Credit: fortiss GmbH/University of Bielefeld)

Researchers at Bielefeld University made numerous video recordings of customers ordering drinks in bars in order to program a bartending robot to interpret the body language of patrons who are ready to order another drink.

In loud bars with low lighting, bartenders rely less on verbal cues than body language to determine which customers are trying to buy a drink.


"Currently, we are working on the robot's ability to recognise when a customer is bidding for its attention," says Professor Dr. Jan de Ruiter of the Psycholinguistics Research Group at Bielefeld University. "Thus, we have studied the process of ordering a drink in real life."

Surprisingly to customers, though perhaps not to bartenders, researchers found that waving or gesturing is not the way to get attention at the bar.

Only one in fifteen customers looked at their wallets to signal that they would like to place an order. Fewer than one in twenty-five customers gestured at the bartender.

The most common and successful signals were more subtle. More than 90 percent of customers positioned themselves right up against the bar counter, facing the counter or the bartender.

Researchers found that customers who did not wish to order a drink instinctively avoided these positions. They sat slightly farther from the edge of the counter, and faced away from the bar or toward their companions.


The bartending robot is part of the EU project 'James' (Joint Action in Multimodal Embodied Systems). The project aims to program the robot, named James, to recognize and display the socially intelligent behaviors humans take for granted.

The robot's head is a tablet computer showing large, cartoon eyes which can establish eye contact with the customers, and a mouth that moves in sync with its speech. James' one-armed torso is fixed behind the bar.

The machine currently requires a precise definition of which signals indicate an order and which do not. Without a proper definition, it will discomfort people by responding inappropriately to their signals.

The latest findings, published in Frontiers in Psychology, have been used to update James's programming. When a customer's body posture indicates a desire to order, James will respond with "How can I help you?" He will also memorize the order in which people signal for his attention, and serve them in order.

Latest Headlines