Robot uses AI to mimic human facial expressions

A robot capable of mimicking human facial expressions has been developed by engineers at Columbia University in the United States.

Engineers at the Creative Machines Lab developed ‘Emo’, a robot face which researchers believe is a significant advancement in non-verbal communication between humans and robots.

Emo, described in a study published in Science Robotics, can anticipate human facial expressions and mimic them simultaneously, even predicting a forthcoming smile around 840 milliseconds before it happens.

The study’s lead author explained how the team faced the challenges of developing a mechanically expressive face and determining when to generate natural, timely expressions.

“The primary goal of the Emo project is to develop a robotic face that can enhance the interactions between humans and robots,” explained PhD student and lead author, Yuhang Hu.

“As robots become more advanced and complicated like those powered by AI models, there’s a growing need to make these interactions more intuitive.”

Emo’s human-like head uses 26 actuators for a range of facial expressions and is covered with silicone skin. It features high-resolution cameras in its eyes for lifelike interactions and eye contact, crucial for nonverbal communication. The team used AI models to predict human facial expressions and generate corresponding robot facial expressions.

The robot was trained using a process termed “self-modeling”, wherein Emo made random movements in front of a camera, learning the correlation between its facial expressions and motor commands. After observing videos of human expressions, Emo could predict people’s facial expressions by noting minor changes as they intend to smile.

The team says the study marks a shift in human-robot interaction, allowing robots to factor in human expressions during interactions, improving interaction quality and fostering trust.

The team plans to integrate verbal communication into Emo to allow the robot “to engage in more complex and natural conversations.”

Watch the video above to see Emo in action and hear more from Yuhang Hu.