Using facial expression analysis during a musical experiment

Using facial expression analysis during a musical experiment

Posted by Jacqueline Martinali on Tue 10 Jul. 2018 - 2 minute read

The influence of music on our emotions intrigues me, and raises many questions. For example, how is it possible that my emotion changes as music speeds up or slows down? Why does the song ‘Happy’ by Pharrell Williams make me smile, and why do I react in the opposite manner to the song ‘Tears in Heaven’ by Eric Clapton?

Is it really the music itself that triggers these emotions, or is it the memory of a particular event that fits with the music or any other association? Why do I prefer to listen to fast songs at one time and to slow songs at another time? Can anyone else around me read my face and figure out what emotion the music that I’m listening to evokes? Does everyone react the same way? And if they don’t, why not?

The link between music and emotion

Many processes that take place in our brains determine how we react to music. Scientist have figured out that music affects people, but to what extent varies per person. For example, the volume of the music has the most influence on emotions. After that, the pace of the music follows. The melody and the instruments play a role as well.

Time to do a small experiment

I decided to do a small experiment using FaceReader Online to measure facial expressions while my participants watched a music video on their own computer or laptop. I chose three different music fragments:

  1. Drummer boy Gordo performing his act on the street, using plastic baskets
  2. A rendition of Beethoven’s “Ode to Joy” by Beaker, Dr. Bunsen Honeydew's loyal assistant from the Muppets
  3. A sad and emotional piano song, purely instrumental

FaceReader Online has an advantage in that you can invite people from all over the world to join your study. Sending a link via an email and starting the experiment was fast and simple. Several people responded, and the recordings simply appeared one by one on my dashboard.

Did the Muppets make people smile?

After easily grouping my participants, the next step was to look at the results. Within FaceReader Online, it is possible to make a quick comparison of your results. For example, you can compare the media files, the participant groups (male/female or based on age or country e.g.), or certain episodes within the media files that you have selected beforehand.

Visualize the comparison of all participants in FaceReader Online

Expression comparison of all participants

The pie chart shows the Expression intensity of the participating males

Expression Intensity of the males

The results mostly confirmed my expectations. The Muppets fragment indeed resulted in the happiest faces. The drummer boy and the piano man mainly triggered neutral facial expressions.

What surprised me was that the piano song showed little sad emotions overall. Maybe in this case, the distinction between a neutral feeling and a sad feeling was not large enough. When comparing males and females, two fragments showed some differences. Males responded with a much higher level of excitement (arousal) to the piano song, and were happier when listening to the drummer boy.

The ease of use of FaceReader Online

FaceReader Online offers an interactive web page allowing you to choose how you want to visualize your results. It is quick, easy, and straightforward to use. For a more detailed analysis, all results can be exported to the 'offline' FaceReader environment. This allows you to review all participant recordings and gives you access to a larger number of analysis tools and visualization options.

As a result of this experiment, I discovered that music and emotions are complex. But, on the other hand, FaceReader Online definitely is not. In fact, it is just the opposite; it is easy to work with, and the online manual is very useful. In my opinion, the convenience with which you can let people from all over the world participate in your research is its greatest advantage.

RESOURCES: Read more about FaceReader

Find out how FaceReader is used in a wide range of studies and how it can elevate your research!

  • Free white papers and case studies
  • Customer success stories
  • Recent blog posts


Don't miss out on the latest blog posts
Share this post
Relevant Blogs

Entrepreneurs’ facial expressions & emotional contagion during investor pitches

What is the influence of emotions in a decision-making process? Dinkla and Liebregts wondered if displaying joy and emotional mimicry play a role during pitches.

SUKIPANI: The magic word for making a smile

The SUKIPANI smile is an exercise to train the muscles you use while smiling. Dr. Sugahara explains the effect of the movements of the muscles and uses FaceReader to analyze the smiles.

Emotional responses to winning a paralympic medal

Guest blogger Jeffrey J. Martin conducted a study to examine the emotional reactions of Paralympians right after winning a medal, using automated facial expression analysis.