Smile like you mean it

Smile like you mean it

Of all human expressions, a smile is the most universal. But can you tell which smile is real and which is false?

Posted by

Jason Rogers

Published on

Thu 06 Sep. 2018

Of all human expressions, a smile is the most universal. It has tremendous power and can even transform your own subjective state. For example, when not in a good mood originally, smiling can increase subjective-rated happiness1. Pretty cool, eh?

But when is a smile not a smile? When it’s literally not that. But when and how can a smile not be a smile? Simple: when it’s false.

Automated facial expression analysis

In the early 80s, Paul Ekman and Wallace Friesen2 proposed that there are two types of smiles: real, or ‘felt’ smiles, and false ones. The real ones are also called Duchenne smiles, named for Guillaume Duchenne, who first characterized the facial muscles needed to make the expression happen. According to their research, felt smiles activate muscles in the eyes and false smiles do not3.

In addition, false smiles have been shown to be more unilateral, meaning the mouth curls more to one side. In the classic image below4, can you tell the difference between the two smiling Pauls?

Smiles paul Ekman

Automated facial expression analysis tools, such as FaceReader5, can not only characterize expressions like happiness, but they can also determine the individual muscles used. The muscles, called Action Units6, are expressed on a categorical scale (ranging from A to E) based on the amount of muscle needed.








FREE WHITE PAPER: FaceReader methodology

Download the free FaceReader methodology note to learn more about facial expression analysis theory.

  • How FaceReader works
  • More about the calibration
  • Insight in quality of analysis & output

FACS Action Units

There are twenty eight main Action Units in the face7, of which FaceReader can classify twenty. Three of those, AU6, AU7, and AU12 are the three muscles needed to express happiness (see here for video). FaceReader detects the presence of a unilateral smile with AU14. Thus, using FaceReader, I wanted to test expressions from those who were really smiling or just faking it.

A video8 was sent to volunteers via the online platform, FaceReader Online9. The video, which was found via YouTube, was a series of clips of ‘fails’, which is to say, situations of physical comedy. The results were exported to FaceReader 7.1 in order to classify the Action Units.

Finally, data were exported to The Observer XT10 for segmentation and detailed analysis. Each respondent video was subsequently classified as ‘neutral’, ‘real’, or ‘false’ smiling. Data were segmented blindly, which is to say that I had no idea what muscles were active. I simply coded ‘real’ and ‘false’ based on my own expertise and knowledge of faces. Finally, the Action Unit data was exported as a percentage of time in which the AUs were active.

Real of false smiles?

The hypothesis was simple: real smiles will show more AU6 and AU7, whereas false smiles will show more AU14. In addition, the categorical expression of the AUs will be stronger in the eyes of real smiles.

Not surprisingly, that’s what I found. Real smiles showed more AU6 and AU7, whereas false smiles showed more AU14. In addition, the AUs themselves were stronger with their respective smile.

Chart real smiles vs false smiles
chart cheek raiser lid tightener real and false
chart lip corner puller dimpler real and false

It can truly be said that you smile with your eyes. What does that mean for you and your business? Can your interviewer tell the difference between that felt or false smile when the respondent is describing her experience with your product? Maybe your respondent isn’t sure if she liked that sample, so she’s just telling you what she thinks you want to hear.

Furthermore, what about your facial expression analysis tool: Can it tell the difference? Smiling Paul was a warm-up. Let’s try another example. In the image below, can you tell which of Dominic’s smiles is real and which is false?

Smiles Dominic false and real smile

What if I made it even easier and showed you the AUs? Remember, AU6 and AU7 vs AU14:

Smiles Dominic false and real smile with facereader

You guessed it. It was the one on the right. Interestingly, FaceReader reads Dominic as happy in both images; however, only one smile was real.

So when you ask a respondent about your brand, product, or service, be sure to check their eyes. Can’t tell the difference or don’t want to have someone trained to do it? Try FaceReader with Action Unit Module. The Eagles may have said it best when they wrote, “You can’t hide your lying eyes.”

I struggled with this for many years. I have a very hard time showing a real smile in pictures. It has gotten better, but only after my friends finally told me that I looked bored and angry in pictures. The breaking point was one person commented that my face was ruining the shot! It was a joke, but many a truth is said in jest.

To fix the issue, and to keep my friends, I spent many years practicing in front of the mirror. The issue was simple: I couldn’t fake it. It seems that in order to express happiness, I must actually be happy! The practicing paid off. Now, whenever I’m forced into having my picture taken, I remember my mirror training. But now, like Harry Potter with his stag, I conjure my Petronus11 and you get a smile.

See for yourself; progress:

Smiles Jason Rogers Happy

Oh: Still wondering about which Smiling Paul was the correct one? It was the Paul on the left.

References

  1. https://www.psychologytoday.com/us/blog/prefrontal-nudity/201208/smile-powerful-tool
  2. Ekman, P. & Friesen, W.V. J Nonverbal Behav (1982) 6: 238. https://doi.org/10.1007/BF00987191
  3. https://www.psychologytoday.com/us/blog/thriving101/201001/what-science-has-say-about-genuine-vs-fake-smiles
  4. Retrieved from https://sonnemann.files.wordpress.com/2012/05/paul-ekman1.png
  5. https://www.noldus.com/human-behavior-research/products/facereader
  6. https://www.noldus.com/facereader/facial-action-units
  7. https://en.wikipedia.org/wiki/Facial_Action_Coding_System
  8. https://www.youtube.com/watch?v=yRB_UBef-sA&t=124s
  9. https://facereader-online.com/Default.aspx
  10. https://www.noldus.com/human-behavior-research/products/the-observer-xt
  11. https://www.pottermore.com/writing-by-jk-rowling/patronus-charm

Related Posts

Predicting behavior from non-verbal communication
06 Dec human behavior research Psychology

Predicting behavior from non-verbal communication

Suppose a darts player is ready to throw a dart while showing a smile on his face. Would he hit the attempted number of points as opposed to missing the shot if he looked grumpy? Research has been done to figure this out.
What does an infant’s gaze tell us about how hungry they feel?
11 Apr human behavior research Psychology

What does an infant’s gaze tell us about how hungry they feel?

McNally and her colleagues developed a coding scheme to observe infant gaze behavior and applied it in a study of complementary feeding.
Designing a behavioral coding scheme to identify complex trauma
25 Feb human behavior research Psychology

Designing a behavioral coding scheme to identify complex trauma

In order to contribute to the development of diagnostic tools and a better understanding of the effects of complex trauma, Ana Granados and her team proposed a new coding scheme and tested its validity.