Computer Laboratory

Projects

Teaser figure
OpenFace is an open source framework that implements state-of-the-art facial behavior analysis algorithms including: facial landmark detection, head pose tracking, eye gaze and facial Action Unit estimation.

Abstract

Over the past few years, there has been an increased interest in automatic facial behavior analysis and understanding. We present OpenFace – an open source tool intended for computer vision and machine learning researchers, affective computing community and people interested in building interactive applications based on facial behavior analysis. OpenFace is the first open source tool capable of facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation. The computer vision algorithms which represent the core of OpenFace demonstrate state-of-the-art results in all of the above mentioned tasks. Furthermore, our tool is capable of real-time performance and is able to run from a simple webcam without any specialist hardware. Finally, OpenFace allows for easy integration with other applications and devices through a lightweight messaging system.

Downloads

Code

The code available for download uses the approaches described in the paper. However, the code is still in the process of being developend and improved (both for accuracy and speed). Therefore, the results of running the code on the datasets used in the paper are slightly those reported in the paper.

The code also contains experiment running scripts, with the results to expect from running the code on the publicly available datasets used in the paper and described below.

The exact code that was run in the paper can also be made available upon request. However that code is not cleaned up and would be slightly more difficult to follow.

Citing our work

If you use any of the resources provided on this page in any of your publications we ask you to cite the following work.

OpenFace: an open source facial behavior analysis toolkit
Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency
in IEEE Winter Conference on Applications of Computer Vision, Lace Placid, NY, March 2016

Bibtex

@inproceedings{Baltrusaitis2016,
  author	=	{Tadas Baltru\v{s}aitis and Peter Robinson and Louis-Philippe Morency},
  title		=	{OpenFace: an open source facial behavior analysis toolkit},
  booktitle	=	{IEEE Winter Conference on Applications of Computer Vision},
  year		=	2016,
}