How to learn emotions from a game

Thursday, 17 November, 2011

Automatic emotion detection provides valuable insight into human-machine interaction measuring the users’ direct response and is therefore often used in this type of research. It can also contribute to learning processes by providing the test participant with direct feedback about his or her facial expressions.

Shahid, Krahmer and Swerts (2010) present four affective and adaptive games developed under the GamE (Game as a Method for Eliciting Emotions) paradigm.

An example – Emotion Park Game

One game, Emotion Park Game, can be used for inducing and teaching six basic emotions to children.

Think about the possibilities when working with autistic children. Direct feedback provides them with the information they need to match facial expressions to emotions. Often we see that it can be challenging to show the facial expression that corresponds with the emotion. Direct feedback gives children a chance to learn more about their expressions and thus provides them with extra tools they can use in day-to-day communication. Because it’s a game, children are often immediately interested. Moreover, players can only move to the next stage when they display the right emotion. Competition is everything!

Facial recognition software

Shahid and colleagues used facial recognition software to automatically detect and analyze facial expressions (six basic emotions).

By using this software, facial expressions are classified directly in one of the following categories: happy, sad, angry, surprised, scared, disgusted, and neutral. These emotional categories have been described by Ekman (1970) as the basic or universal emotions.  Facial expressions can vary in intensity and are often a mixture of emotions. By placing a camera in front of a test participant, the software can ‘read’ the face. Emotions that aren’t expressed cannot be seen by others. It’s therefore a very important factor in inter-personal communication.

FaceReader

FaceReader methodology white paper

Request the FREE FaceReader methodoloty note to learn more about facial expression analysis theory.

  • Learn what FaceReader is and how it works
  • Learn how the calibration works
  • Get insight in quality of analysis and output

References

  • Ekman, P. (1970). Universal facial expressions of emotion. California Mental Health Research Digest8, 151-158.
  • Shahid, S.; Krahmer, E.; Swerts, M. (2010). GamE Paradigm: Affective gaming for affect elicitation. ACE 2010, Taipei, Taiwan, 17 November 2010.