Of all human expressions, a smile is the most universal. It has tremendous power and can even transform your own subjective state. For example, when not in a good mood originally, smiling can increase subjective-rated happiness1. Pretty cool, eh?
But when is a smile not a smile? When it’s literally not that. But when and how can a smile not be a smile? Simple: when it’s false.
Automated facial expression analysis
In the early 80s, Paul Ekman and Wallace Friesen2 proposed that there are two types of smiles: real, or ‘felt’ smiles, and false ones. The real ones are also called Duchenne smiles, named for Guillaume Duchenne, who first characterized the facial muscles needed to make the expression happen. According to their research, felt smiles activate muscles in the eyes and false smiles do not3.
In addition, false smiles have been shown to be more unilateral, meaning the mouth curls more to one side. In the classic image below4, can you tell the difference between the two smiling Pauls?
Automated facial expression analysis tools, such as FaceReader5, can not only characterize expressions like happiness, but they can also determine the individual muscles used. The muscles, called Action Units6, are expressed on a categorical scale (ranging from A to E) based on the amount of muscle needed.
FaceReader methodology white paper
Request the FREE FaceReader methodology note to learn more about facial expression analysis theory.
- Learn what FaceReader is and how it works
- Learn how the calibration works
- Get insight in quality of analysis and output
FACS Action Units
There are twenty eight main Action Units in the face7, of which FaceReader can classify twenty. Three of those, AU6, AU7, and AU12 are the three muscles needed to express happiness (see here for video). FaceReader detects the presence of a unilateral smile with AU14. Thus, using FaceReader, I wanted to test expressions from those who were really smiling or just faking it.
A video8 was sent to volunteers via the online platform, FaceReader Online9. The video, which was found via YouTube, was a series of clips of ‘fails’, which is to say, situations of physical comedy. The results were exported to FaceReader 7.1 in order to classify the Action Units.
Finally, data were exported to The Observer XT10 for segmentation and detailed analysis. Each respondent video was subsequently classified as ‘neutral’, ‘real’, or ‘false’ smiling. Data were segmented blindly, which is to say that I had no idea what muscles were active. I simply coded ‘real’ and ‘false’ based on my own expertise and knowledge of faces. Finally, the Action Unit data was exported as a percentage of time in which the AUs were active.
Real of false smiles?
The hypothesis was simple: real smiles will show more AU6 and AU7, whereas false smiles will show more AU14. In addition, the categorical expression of the AUs will be stronger in the eyes of real smiles.
Not surprisingly, that’s what I found. Real smiles showed more AU6 and AU7, whereas false smiles showed more AU14. In addition, the AUs themselves were stronger with their respective smile.
It can truly be said that you smile with your eyes. What does that mean for you and your business? Can your interviewer tell the difference between that felt or false smile when the respondent is describing her experience with your product? Maybe your respondent isn’t sure if she liked that sample, so she’s just telling you what she thinks you want to hear.
Furthermore, what about your facial expression analysis tool: Can it tell the difference? Smiling Paul was a warm-up. Let’s try another example. In the image below, can you tell which of Dominic’s smiles is real and which is false?
What if I made it even easier and showed you the AUs? Remember, AU6 and AU7 vs AU14:
You guessed it. It was the one on the right. Interestingly, FaceReader reads Dominic as happy in both images; however, only one smile was real.
So when you ask a respondent about your brand, product, or service, be sure to check their eyes. Can’t tell the difference or don’t want to have someone trained to do it? Try FaceReader with Action Unit Module. The Eagles may have said it best when they wrote, “You can’t hide your lying eyes.”
I struggled with this for many years. I have a very hard time showing a real smile in pictures. It has gotten better, but only after my friends finally told me that I looked bored and angry in pictures. The breaking point was one person commented that my face was ruining the shot! It was a joke, but many a truth is said in jest.
To fix the issue, and to keep my friends, I spent many years practicing in front of the mirror. The issue was simple: I couldn’t fake it. It seems that in order to express happiness, I must actually be happy! The practicing paid off. Now, whenever I’m forced into having my picture taken, I remember my mirror training. But now, like Harry Potter with his stag, I conjure my Petronus11 and you get a smile.
See for yourself; progress:
Oh: Still wondering about which Smiling Paul was the correct one? It was the Paul on the left.
- Ekman, P. & Friesen, W.V. J Nonverbal Behav (1982) 6: 238. https://doi.org/10.1007/BF00987191
- Retrieved from https://sonnemann.files.wordpress.com/2012/05/paul-ekman1.png