Computer Laboratory

Course pages 2017–18

Affective Computing

Principal lecturer: Dr Hatice Gunes
Taken by: MPhil ACS, Part III
Code: L44
Hours: 16
Class limit: 20 students
Prerequisites: Matlab programming skills, basic background in signal/image processing and/or machine learning desirable

Aims

Understanding, (automatically) analysing and modelling people's affective and social behaviour is very important for multiple domains such as enhancing human-computer and human-agent interaction, improving gaming technology and players’ experience, behavioural analytics for the banking sector, etc. Accordingly, the aim of this module is to impart knowledge and ability needed to make informed choices of models, data, and techniques for sensing, recognition, and expression of human affective and social behaviour (e.g., smile, frown, head nodding/shaking, agreement/disagreement), and its use in the design of innovative interactive technology (e.g., interaction with virtual agents, robots, and games; single and multi-user smart environments, e.g., in a car; implicit (multimedia) tagging; clinical and biomedical studies, e.g., autism, depression, pain).

Syllabus

The following list provides a representative list of topics:

  • Definitions, Overview and Applications: entertainment technology/gaming/arts; clinical and biomedical studies, e.g., autism, depression, pain; etc.; implicit (multimedia) tagging; affective wearables);
  • Emotion theories: psychology, cognitive science and neuroscience;
  • Sensing: vision, audio, bio signals, text; data acquisition and annotation, tools and databases;
  • Processing and feature extraction: extracting meaningful information and features from multiple modalities;
  • Recognition, Prediction and Evaluation: applying various machine learning techniques; commonly utilised evaluation strategies;
  • Synthesis: Affect and expression synthesis and generation (virtual characters, robots, etc.);
  • Emotional design: frameworks (e.g., visceral, behavioural, reflective design)
  • Advanced topics and Ethical considerations;
  • Hands-on programming work with relevant libraries relevant for affective and behavioural computing.

Objectives

The aim of this module is to impart knowledge and ability needed to make informed choices of models, data, and techniques for sensing, recognition, and expression of human affective and social behaviour (e.g., smile, frown, head nodding/shaking, joy/depression), and its use in the design of innovative interactive technology (e.g., interaction with virtual agents, robots, and games; single and multi-user smart environments, e.g., in a car; implicit (multimedia) tagging; clinical and biomedical studies, e.g., autism, depression, pain).

On completion of this module, students should:

  • Understand the challenges in human-human affective and communicative interaction (e.g. not what is said but how it is said – using the body, head, face, intonation, etc.) and its implication to Human-Computer Interaction;
  • Demonstrate knowledge in current theories and trends in designing emotionally and socially sensitive interactive technology, as well as recent advances in human audio/visual/bio signal processing, and recognition using machine learning and pattern recognition techniques;
  • Comprehend and apply (appropriate) methods for collection, analysis, representation and evaluation of human affective and communicative behaviour data;
  • Demonstrate ability to computationally analyse, recognise and evaluate human affective and social behaviour;
  • Enhance programming skills for human affect and behaviour analysis and understanding;
  • Demonstrate critical thinking, analysis and synthesis while making a decision on 'when' and 'how' to incorporate emotions and social signals in a specific application context, and gain practical experience in proposing and justifying computational solution(s) of suitable nature and scope.

Assessment


Practicals: 10%
Seminars: 25%
Project report: 65% (written report)

Recommended reading

Picard, R. (2000). Affective Computing. MIT Press.
Gokcay, D. & Yildirim, G. (Eds)(2011). Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives.. IGI Global, http://www.igi-global.com/book/affective-computing-interaction/41787.
Salah, A. A. & Gevers, T. (Eds)(2011). Computer Analysis of Human Behavior.. 2011, XV, 409 p. http://www.springer.com/computer/hci/book/978-0-85729-993-2.
Petta, P., Pelachaud, C. & Cowie, R. (Eds) (2011). Emotion-Oriented Systems: The Humaine Handbook.. Springer Verlag. Amazon link
Journals:
1) IEEE Transactions on Affective Computing http://www.computer.org/portal/web/tac/about
2) IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=3477
3) International Journal of Synthetic Emotions http://www.igi-global.com/journal/international-journal-synthetic-emotions-ijse/1144
Conference proceedings:
1) ACII: Affective Computing and Intelligent Interaction
2) ICMI: ACM International Conference on Multimodal Interaction
3) FGR: IEEE Conference on Automatic Face and Gesture Recognition
Online Resources:
1) The HUMAINE Portal: http://emotion-research.net/
2) The SSPNet Portal: http://sspnet.eu/