Insights

What did we study?

A key element of product development is understanding the user. In design interviews, the designer tries to understand what the user thinks and feels, and what are their needs.Correctly identifying these needs helps to create useful and valuable products for the users. But how can we measure, if the designer really understands the user?

Our approach was to measure facial synchrony between the user and the designer, and test if the synchrony indicates whether the designer is understanding the user or not. Synchrony is a phenomenon, where a person mirrors another person’s movements, vocal tones, and facial expressions. Synchrony is a physiological form of empathy, where emotions transfer from one person to another through autonomous movements. For example, when you are happy and smile at a friend, they will automatically smile back at you and just might get happy as well.

Test setup

We conducted 46 video-recorded design interviews, where designers talked with users about their driving experiences. The recordings were conducted on zoom, because of the current Covid-19 situation. After the interviews, the designers performed a written psychological task, that measured how accurately they can guess the user’s thoughts and emotional tones.

We also ran a facial expression analysis on the video recordings and measured the synchrony between the user and the designer during the interviews. We observed the movements in the facial muscles of the user and the designer and analyzed whether they are synchronized or not.

Results

Is there synchrony between the user and the designer?

There is synchrony between the user and the designer. Synchrony occurred when the user and the designer were smiling. However, for negative expressions, such as frowning, there was no statistically significant synchrony. Smiling synchrony often happened so that the user first smiled, and then the designer mirrored the user’s smile.

Was there a link between the synchrony and the designer’s performance?

During the parts of videos, that the user marked as feeling positive there was a link between the smiling synchrony and how well the designer guessed the user’s emotional tone. When the user and the designer were not smiling, the designer was not likely to guess that the user’s emotional tone is positive. When the user and the designer were smiling, the designer was most likely to guess correctly, that the user actually is positive.

Figure: The designer guesses the user’s emotional tone more accurately when there is synchrony.

The results suggest that we can use facial expression analysis to observe the synchrony between a designer and a user. This synchrony also seems to have a relationship to how well the designer understands the user, but this relationship will be analyzed deeper in the next phases of the Empathic Engineers project.