Determine facial expressions in three steps with FaceReader
- Face finding – an accurate position of the face is found using the popular Viola-Jones algorithm.
- Face modeling – the Active Appearance Model is used to synchronize an artificial face model, which describes the location of over 500 key points as well as the texture of the face. These outcomes are combined with the results of the Deep Face algorithm to achieve a higher classification accuracy. When face modeling is not successful (for example, when a hand is covering the mouth, but both eyes can be found), the Deep Face algorithm, based on Deep Learning, takes over.
- Face classification – output is presented as seven basic expressions and one neutral state through the artificial neural network.
To save you valuable time when analyzing videos, FaceReader also automatically classifies:
- mouth open-closed
- eyes open-shut
- eyebrows raised-neutral-lowered
- head orientation
- gaze direction
- characteristics: gender, age, and facial hair (beard and/or moustache)
Other independent variables can be entered manually.
Define your own Custom Expressions
Custom Expressions are facial expressions or mental states that you can define yourself by combining the facial expressions and Action Units that FaceReader can recognize. You can also use Valence, Arousal, Head Orientation (Action Unit 51-56), Heart Rate and Heart Rate Variability in your definition of a Custom Expression.
The Custom Expression function can be used for a wide range of applications, for example:
- Define ‘Duchenne smile’ as a Custom Expression by combining Action Unit 6 and 12*
- Analyze the positive expression ‘Awe’ using a combination of Action Unit 1, 5, 25 and 26*
- Research related to pain, based on Action Unit 4, 6, 7, 9, 10, 25, 26, 27, 43*
- If you would like to change the name of an expression, e.g. ‘Smile’ instead of ‘Happy’
- Modify the intensity of an expression
- Measure affective states: Interest, boredom and confusion are already provided as examples when you purchase the Action Unit Module
*These examples are available upon request. Please contact us if you are interested to test or validate these. If you have developed your own custom expression and would like to share this with other researchers, please let us know!
Participant emotion analysis
Facial expressions can be visualized as bar graphs, in a pie chart, and as a continuous signal. A gauge display summarizes the negativity or positivity of the emotion (valence). The timeline gives you a detailed visual representation of the data.
A separate reporting window displays a pie chart with percentages, a smiley, and a traffic light, indicating whether a person’s mood is positive, neutral, or negative. All visualizations are given to you in real-time and may be viewed afterwards. With the Project Analysis Module, advanced facial expression analysis has become available in FaceReader.
Deep Learning: analyze faces under challenging circumstances
With the classification engine Deep Face Model, FaceReader can make sense of large amounts of complex data. What does the Deep Face Model do exactly?
The Deep Face Model makes use of deep learning, which is based on an artificial neural network with multiple layers between the input and the output. The network moves through the layers calculating the probability of each output.
Currently it is the most successful artificial intelligence technique in machine learning. Like in real neural networks, information on the input side is collected and processed by neurons that are connected with each other. Mapping of input to output goes via a series of nonlinear computations, clubbing together lower levels of information to form higher level features (e.g. expressed emotion, age, gender).
Privacy & ethics
FaceReader is installed on-site and adheres to strict privacy-by-design protocols. For example, the software offers you the option not to record the test participants face during the analysis. In this case only metadata are acquired from the recordings of the face that cannot be related to an identifiable person. Examples of metadata are facial expressions, head pose, age and gender.