Computer Laboratory

Technical reports

Affect inference in learning environments: a functional view of facial affect analysis using naturalistic data

Shazia Afzal

December 2010, 146 pages

This technical report is based on a dissertation submitted May 2010 by the author for the degree of Doctor of Philosophy to the University of Cambridge, Murray Edwards College.


This research takes an application-oriented stance on affective computing and addresses the problem of automatic affect inference within learning technologies. It draws from the growing understanding of the centrality of emotion in the learning process and the fact that, as yet, this crucial link is not addressed in the design of learning technologies. This dissertation specifically focuses on examining the utility of facial affect analysis to model the affective state of a learner in a one-on-one learning setting.

Although facial affect analysis using posed or acted data has been studied in great detail for a couple of decades now, research using naturalistic data is still a challenging problem. The challenges are derived from the complexity in conceptualising affect, the methodological and technical difficulties in measuring it, and the emergent ethical concerns in realising automatic affect inference by computers. However, as the context of this research is derived from, and relates to, a real-world application environment, it is based entirely on naturalistic data. The whole pipeline – of identifying the requirements, to collection of data, to the development of an annotation protocol, to labelling of data, and the eventual analyses – both quantitative and qualitative; is described in this dissertation. In effect, a framework for conducting research using natural data is set out and the challenges encountered at each stage identified.

Apart from the challenges associated with the perception and measurement of affect, this research emphasises that there are additional issues that require due consideration by virtue of the application context. As such, in light of the discussed observations and results, this research concludes that we need to understand the nature and expression of emotion in the context of technology use, and pursue creative exploration of what is perhaps a qualitatively different form of emotion expression and communication.

Full text

PDF (3.0 MB)

BibTeX record

  author =	 {Afzal, Shazia},
  title = 	 {{Affect inference in learning environments: a functional
         	   view of facial affect analysis using naturalistic data}},
  year = 	 2010,
  month = 	 dec,
  url = 	 {},
  institution =  {University of Cambridge, Computer Laboratory},
  number = 	 {UCAM-CL-TR-793}