Why so angry? The role of context and function in facial expression analysis
In a previous blog post titled “How emotions are made”, I outlined how neuroscience research in the past decades has shown that our brain gives meaning to our experiences/sensations through concepts such as emotions. So, emotions are created in the brain, rather than triggered by fixed brain circuitry in response to specific stimuli.
I ended the blog with the question: “Why do respondents sometimes show an angry facial expression when evaluating a website or product, when there is no reason to expect them to actually be angry”? In the current blog post I will try to answer this question.
Hint: context and function.
Facial expressions have a function in social communication. Humans can ‘read’ a fellow human being’s face to infer the mental state of this other person. This might result in the receiver experiencing and sharing the same emotional state as the sender. Subsequently, this can evoke some kind of action in the receiver, depending on what the emotional state and the circumstances require.
'The Expression of the Emotions' by Charles Darwin
There is also a different but important aspect to facial expressions that was already proposed in 1872 by Charles Darwin in his book The Expression of the Emotions in Man and Animals (1).
Darwin suggested that facial expressions do not originate in social communication, but initially have evolved as functional adaptations to modify an organisms perception and prepare for action in response to specific stimuli.
Let’s look at an example
When an early human encountered a potential dangerous situation in the form of a predator it was adaptive to enhance vigilance and perception. This is exactly what happens with a facial expression of fear: the face shows a general ‘expressive widening’ with eye, nose/mouth opening and brow raising, resulting in a better perception of visual stimuli.
[Adapted from Figure 2A, page 844, Susskind et al, 2008, Nature Neuroscience (2)]
Principle of form
Besides this principle of function, Darwin also proposed the principle of form. According to this principle, emotional states with opposite functions also have opposing facial actions. The example of the principle of form that Darwin gives in The Expression of the Emotions in Man and Animals is that of a dog in two opposing states.
In one drawing a dog is shown in a dominant, hostile state. The whole body is upright, head raised, ears pricked, tail is held high and the hairs are raised, especially along the neck and back.
In contrast, a dog in a submissive, humble state displays an opposing body posture: the body and head are held low, ears are lowered, the tail is held low and sometimes retracted underneath the body.
Investigate fear and disgust
These principles of function and form in relation to facial expressions were studied by Susskind et al (2). They looked at fear and disgust, as opposing emotional states, because fear is about sensory vigilance whereas disgust is about sensory rejection. They set about to investigate if this is reflected in differences in the facial expressions associated with fear and disgust.
Susskind and coworkers first looked at the shape of the face using vector representations in a computer graphics model of facial appearance. They found that when they created an antiprototype of the classic fear expression, participants reported that it most closely resembled a disgust expression.
When they created an antiprototype of disgust, it most closely resembled a fear expression. This is also displayed in the opposing vector representations of fear (A in the picture below) and disgust (B in the picture below): raised versus lowered brows, increased versus decreased eye aperture and vertical elongation versus compression of the nose associated with raised versus lowered lips.
Next, they asked participants to judge the size of their visual field relative to posing neutral expressions during a facial-action task in which participants posed both fear and disgust. From the picture below it is clear that the subjective visual field was larger in a fear expression compared to a disgust expression (with a neutral expression in between). The decreased eye aperture in the disgust expression allows the person to better see proximate stimuli that need close inspection before rejection.
They also tested whether sensory rejection, associated with facial expression of disgust, was reflected in a different olfactory intake from a facial expression of fear. They indeed found that the expression configuration had a significant effect on air intake, increasing from disgust to neutral to fear.
Read more about the possibilities and limitations of FaceReader in our facts about FaceReader blog post.
Is it real anger?
How do these results affect the interpretation of the angry expression on someone’s face who is looking at a website? At the beginning of this blog post, I gave two hints.
The first one was context. In the context of a website, it is unlikely that this creates strong emotions in a participant (of course, depending on the contents of the website). Therefore, the angry facial expression might be better explained by the second hint: function.
One of the main characteristics of an angry facial expression is brow lowering combined with more closed eyes. This is also part of the disgust expression, where we saw that functionally this allows a person to better focus on stimuli close-by. In the website context, the function of the brow lowering + eye closing is to better be able to concentrate / focus on the contents of the website.
Why would I make this face?
So, next time you are puzzled by FaceReader’s output, think about context and function. Is the emotion detected appropriate within the context of your study? Or is there an individual muscle/AU driving that ‘emotion’ that is likely to blame? For that, you must consider the functional outcome of the expression.
Try it for yourself: Make the same face that you recorded. Ask yourself, ‘Why would I make this face?’ Then you might catch a glimpse of what your brain has just created for you.
Get the latest blog posts delivered to your inbox - every 15th of the month
more
Disclosure dilemma
Isabelle Leenders examined the influence of disclosures in influencer advertising on the brand attitudes and purchase intentions of young adults, mediated by their emotions and activation of persuasion knowledge.
AI and deep learning: Responsible use of facial expression analysis
The European Parliament adopted its negotiating position on the first-ever regulation of AI. In this blog post, we explore how to use facial expression analysis responsibly and how it can contribute to scientific research.