Smile like you mean it

Smile like you mean it

Posted by Jason Rogers on Thu 06 Sep. 2018 - 4 minute read

Of all human expressions, a smile is the most universal. It has tremendous power and can even transform your own subjective state. For example, when not in a good mood originally, smiling can increase subjective-rated happiness1. Pretty cool, eh?

But when is a smile not a smile? When it’s literally not that. But when and how can a smile not be a smile? Simple: when it’s false.

Automated facial expression analysis

In the early 80s, Paul Ekman and Wallace Friesen2 proposed that there are two types of smiles: real, or ‘felt’ smiles, and false ones. The real ones are also called Duchenne smiles, named for Guillaume Duchenne, who first characterized the facial muscles needed to make the expression happen. According to their research, felt smiles activate muscles in the eyes and false smiles do not3.

In addition, false smiles have been shown to be more unilateral, meaning the mouth curls more to one side. In the classic image below4, can you tell the difference between the two smiling Pauls?


Smiles paul Ekman

Automated facial expression analysis tools, such as FaceReader5, can not only characterize expressions like happiness, but they can also determine the individual muscles used. The muscles, called Action Units6, are expressed on a categorical scale (ranging from A to E) based on the amount of muscle needed.









FREE WHITE PAPER: FaceReader methodology

Download the free FaceReader methodology note to learn more about facial expression analysis theory.

  • How FaceReader works
  • More about the calibration
  • Insight in quality of analysis & output


FACS Action Units

There are twenty eight main Action Units in the face7, of which FaceReader can classify twenty. Three of those, AU6, AU7, and AU12 are the three muscles needed to express happiness (see here for video). FaceReader detects the presence of a unilateral smile with AU14. Thus, using FaceReader, I wanted to test expressions from those who were really smiling or just faking it.

A video8 was sent to volunteers via the online platform, FaceReader Online9. The video, which was found via YouTube, was a series of clips of ‘fails’, which is to say, situations of physical comedy. The results were exported to FaceReader 7.1 in order to classify the Action Units.

Finally, data were exported to The Observer XT10 for segmentation and detailed analysis. Each respondent video was subsequently classified as ‘neutral’, ‘real’, or ‘false’ smiling. Data were segmented blindly, which is to say that I had no idea what muscles were active. I simply coded ‘real’ and ‘false’ based on my own expertise and knowledge of faces. Finally, the Action Unit data was exported as a percentage of time in which the AUs were active.

Real of false smiles?

The hypothesis was simple: real smiles will show more AU6 and AU7, whereas false smiles will show more AU14. In addition, the categorical expression of the AUs will be stronger in the eyes of real smiles.

Not surprisingly, that’s what I found. Real smiles showed more AU6 and AU7, whereas false smiles showed more AU14. In addition, the AUs themselves were stronger with their respective smile.


Chart real smiles vs false smiles


chart cheek raiser lid tightener real and false


chart lip corner puller dimpler real and false

It can truly be said that you smile with your eyes. What does that mean for you and your business? Can your interviewer tell the difference between that felt or false smile when the respondent is describing her experience with your product? Maybe your respondent isn’t sure if she liked that sample, so she’s just telling you what she thinks you want to hear.

Furthermore, what about your facial expression analysis tool: Can it tell the difference? Smiling Paul was a warm-up. Let’s try another example. In the image below, can you tell which of Dominic’s smiles is real and which is false?


Smiles Dominic false and real smile

What if I made it even easier and showed you the AUs? Remember, AU6 and AU7 vs AU14:


Smiles Dominic false and real smile with facereader

You guessed it. It was the one on the right. Interestingly, FaceReader reads Dominic as happy in both images; however, only one smile was real.

So when you ask a respondent about your brand, product, or service, be sure to check their eyes. Can’t tell the difference or don’t want to have someone trained to do it? Try FaceReader with Action Unit Module. The Eagles may have said it best when they wrote, “You can’t hide your lying eyes.”

I struggled with this for many years. I have a very hard time showing a real smile in pictures. It has gotten better, but only after my friends finally told me that I looked bored and angry in pictures. The breaking point was one person commented that my face was ruining the shot! It was a joke, but many a truth is said in jest.

To fix the issue, and to keep my friends, I spent many years practicing in front of the mirror. The issue was simple: I couldn’t fake it. It seems that in order to express happiness, I must actually be happy! The practicing paid off. Now, whenever I’m forced into having my picture taken, I remember my mirror training. But now, like Harry Potter with his stag, I conjure my Petronus11 and you get a smile.

See for yourself; progress:


Smiles Jason Rogers Happy

Oh: Still wondering about which Smiling Paul was the correct one? It was the Paul on the left.

References

  1. https://www.psychologytoday.com/us/blog/prefrontal-nudity/201208/smile-powerful-tool
  2. Ekman, P. & Friesen, W.V. J Nonverbal Behav (1982) 6: 238. https://doi.org/10.1007/BF00987191
  3. https://www.psychologytoday.com/us/blog/thriving101/201001/what-science-has-say-about-genuine-vs-fake-smiles
  4. Retrieved from https://sonnemann.files.wordpress.com/2012/05/paul-ekman1.png
  5. https://www.noldus.com/human-behavior-research/products/facereader
  6. https://www.noldus.com/facereader/facial-action-units
  7. https://en.wikipedia.org/wiki/Facial_Action_Coding_System
  8. https://www.youtube.com/watch?v=yRB_UBef-sA&t=124s
  9. https://facereader-online.com/Default.aspx
  10. https://www.noldus.com/human-behavior-research/products/the-observer-xt
  11. https://www.pottermore.com/writing-by-jk-rowling/patronus-charm
Don't miss out on the latest blog posts
Share this post
Topics
Learn
more
Relevant Blogs
observing-and-coding-behavior-siblings

Observing and coding the behavior of siblings

Would it help if your brother or sister is sitting by your side when you're in pain? Would that ease the pain, or provide some distraction?
parent-child-interaction-effectiveness

Parent-child interaction: Measuring the effectiveness of interactions

Quality of parent-child interaction is one of the major predictors of emotional and social development of children, which makes parent-child interaction an important focus of child development research.
parenting-practices-nutrition-risk

How do parenting practices relate to children’s nutrition risk?

Nutrition plays a vital role in children’s growth, development, and health. Researcher Walton and her team observed family mealtimes to gain a better understanding of children’s nutrition risk.