skip to primary navigationskip to content

Department of Computer Science and Technology



Course pages 2023–24

Affective Computing

Principal lecturer: Prof Hatice Gunes
Taken by: MPhil ACS, Part III
Code: L44
Term: Lent
Hours: 16
Format: In-person lectures
Class limit: max. 20 students
Prerequisites: Python programming skills (other programming languages are also OK but not always ideal), and basic background in machine learning or signal/image processing
Moodle, timetable


Computationally analysing and modelling people's socio-emotional behaviours is very important for multiple domains such as enhancing human-AI, human-agent and human-robot interactions; creating personalized learning environments, behavioural analytics for assessing and improving people’s comfort, healthcare and wellbeing; designing engaging and adaptive training environments and games, etc.

Accordingly, the aim of this module is to impart knowledge and ability needed to make informed choices of models, data, and machine learning techniques for sensing, recognition, and generation of affective and social behaviour (e.g., smile, frown, head nodding/shaking, agreement/disagreement), and its use in the design of innovative interactive technology (e.g., interaction with virtual agents, robots, and games; single and multi-user smart environments, e.g., in-car/ virtual / augmented reality, for public speaking and cognitive training; clinical and biomedical studies, e.g., autism, depression, pain) while addressing the ethical issues (e.g., privacy, bias) arising from the real-world deployment of these systems


The following list provides a representative list of topics:

  • Introduction, definitions, and overview
  • Emotion theories
  • Sensing from multiple modalities (e.g., vision, audio, bio signals, text)
  • Data acquisition and annotation
  • Signal processing / feature extraction
  • Automatic recognition / prediction and evaluation
  • Behaviour synthesis / generation (e.g., for embodied agents / robots)
  • Emotional design frameworks
  • Advanced topics and ethical considerations (e.g., bias and fairness)
  • Applications (via seminar presentations and discussions)
  • Guest lectures (various topics - e.g., commercialising affective computing products)
  • Hands-on programming work (i.e., mini-project)


On completion of this module, students will:

  • Understand the challenges in human-human affective and communicative interaction (e.g. not
    what is said but how it is said – using the body, head, face, intonation, etc.) and its implication to
    human-computer interaction;
  • Demonstrate knowledge in current theories and trends in designing emotionally and socially
    sensitive interactive technology, as well as recent advances in human audio/visual/bio signal
    processing, and recognition using machine learning techniques;
  • Comprehend and apply (appropriate) methods for collection, analysis, representation and
    evaluation of human affective and communicative behaviour data;
  • Demonstrate ability to computationally analyse, recognise and evaluate human affective and
    social behaviour;
  • Enhance programming skills for human affect and behaviour analysis and understanding;
  • Demonstrate critical thinking, analysis and synthesis while making a decision on 'when' and
    'how' to incorporate emotions and social signals in a specific application context, and gain
    practical experience in proposing and justifying computational solution(s) of suitable nature and


Seminar presentation: 20%
Participating in Q&A and discussions: 10%
Mini-Project: 70% (proposal, mid-term written report, final written report, code and presentation)

Recommended reading

Picard, R. (2000). Affective Computing. MIT Press.

Jeon, M. (2017). Emotions and Affect in Human Factors and Human-Computer Interaction. Academic Press.

Calvo, R., D'Mello, S., Gratch, J. and Kappas, A. (2014) The Oxford Handbook of Affective Computing. Oxford University Press.

Tian, L., Oviatt, S., Muszynski, M., Chamberlain, B. C., Healey, J. & Sano, A. (2022) Applied Affective Computing. Association for Computing Machinery, New York, United States.


  1. IEEE Transactions on Affective Computing

Conference proceedings:

  1. ACII: Affective Computing and Intelligent Interaction
  2. ICMI: ACM International Conference on Multimodal Interaction
  3. FGR: IEEE Conference on Automatic Face and Gesture Recognition