Computer Laboratory

Facial Expression Analysis

Just tracking faces is not enough, we also want to understand what the underlying expressions mean. I have explored several ways of using automatic methods for understanding human mental states. I am especially interested in modeling dimensional emotions (valence, power, expectancy and activation) as a regression problem.

Relevant publications

  • What really matters? A study into people's instinctive evaluation metrics for continuous emotion prediction in music
    Vaiva Imbrasaitė, Tadas Baltrušaitis, and Peter Robinson
    in Affective Computing and Intelligent Interaction, Geneva, Switzerland, September 2013
  • Emotion tracking in music using continuous conditional random fields and baseline feature representation
    Vaiva Imbrasaitė, Tadas Baltrušaitis, and Peter Robinson
    in ICME 2013 Workshop on Affective Analysis in Multimedia, San Jose, CA, July 2013
  • Dimensional Affect Recognition using Continuous Conditional Random Fields
    Tadas Baltrušaitis, Ntombikayise Banda, and Peter Robinson
    in IEEE Int. Conf. Automatic Face and Gesture Recognition, Shanghai, China, April 2013
    [pdf][appendix][project page with code to appear]
  • 3D corpus of spontaneous complex mental states
    Marwa Mahmoud, Tadas Baltrušaitis, and Peter Robinson
    in International Conference on Affective Computing and Intelligent Interaction, Memphis, TN, October 2011
  • Modeling Latent Discriminative Dynamic of Multi-Dimensional Affective Signals
    Geovany A. Ramirez, Tadas Baltrušaitis, and Louis-Philippe Morency
    in 1st International Audio/Visual Emotion Challenge and Workshop in conjunction with ACII 2011, Memphis, TN, October 2011
    [Winner of the video sub-challenge]
  • Real-time inference of mental states from facial expressions and upper body gestures
    Tadas Baltrušaitis, Daniel McDuff, Ntombikayise Banda, Marwa Mahmoud, Rana el Kaliouby, Rosalind Picard, and Peter Robinson
    in IEEE Intl Conf. Automatic Face and Gesture Recognition, Facial Expression Recognition and Analysis Challenge
    [pdf]