Affective Computing
Principal lecturer: Prof Hatice Gunes
Taken by: MPhil ACS, Part III
Code: L44
Term: Lent
Hours: 16
Format: In-person lectures
Class limit: max. 20 students
Prerequisites: Python programming skills (other programming languages are also OK but not always ideal), and basic background in machine learning or signal/image processing
Moodle, timetable
Aims
Computationally analysing and modelling people's socio-emotional behaviours is very important for multiple domains such as enhancing human-AI, human-agent and human-robot interactions; creating personalized learning environments, behavioural analytics for assessing and improving people’s comfort, healthcare and wellbeing; designing engaging and adaptive training environments and games, etc.
Accordingly, the aim of this module is to impart knowledge and ability needed to make informed choices of models, data, and machine learning techniques for sensing, recognition, and generation of affective and social behaviour (e.g., smile, frown, head nodding/shaking, agreement/disagreement), and its use in the design of innovative interactive technology (e.g., interaction with virtual agents, robots, and games; single and multi-user smart environments, e.g., in-car/ virtual / augmented reality, for public speaking and cognitive training; clinical and biomedical studies, e.g., autism, depression, pain) while addressing the ethical issues (e.g., privacy, bias) arising from the real-world deployment of these systems
Syllabus
The following list provides a representative list of topics:
- Introduction, definitions, and overview
- Emotion theories
- Sensing from multiple modalities (e.g., vision, audio, bio signals, text)
- Data acquisition and annotation
- Signal processing / feature extraction
- Automatic recognition / prediction and evaluation
- Behaviour synthesis / generation (e.g., for embodied agents / robots)
- Emotional design frameworks
- Advanced topics and ethical considerations (e.g., bias and fairness)
- Applications (via seminar presentations and discussions)
- Guest lectures (various topics - e.g., commercialising affective computing products)
- Hands-on programming work (i.e., mini-project)
Objectives
On completion of this module, students will:
- Understand the challenges in human-human affective and
communicative interaction (e.g. not
what is said but how it is said – using the body, head, face, intonation, etc.) and its implication to
human-computer interaction; - Demonstrate knowledge in current theories and trends in
designing emotionally and socially
sensitive interactive technology, as well as recent advances in human audio/visual/bio signal
processing, and recognition using machine learning techniques; - Comprehend and apply (appropriate) methods for collection,
analysis, representation and
evaluation of human affective and communicative behaviour data; - Demonstrate ability to computationally analyse, recognise
and evaluate human affective and
social behaviour; - Enhance programming skills for human affect and behaviour analysis and understanding;
- Demonstrate critical thinking, analysis and synthesis while
making a decision on 'when' and
'how' to incorporate emotions and social signals in a specific application context, and gain
practical experience in proposing and justifying computational solution(s) of suitable nature and
scope.
Assessment
Seminar presentation: 20%
Participating in Q&A and discussions: 10%
Mini-Project: 70% (proposal, mid-term written report, final
written report, code and presentation)
Recommended reading
Picard, R. (2000). Affective Computing. MIT Press.
Jeon, M. (2017). Emotions and Affect in Human Factors and Human-Computer Interaction. Academic Press. https://www.elsevier.com/books/emotions-and-affect-in-human-factors-and-human-computer-interaction/jeon/978-0-12-801851-4
Calvo, R., D'Mello, S., Gratch, J. and Kappas, A. (2014) The Oxford Handbook of Affective Computing. Oxford University Press. https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199942237.001.0001/oxfordhb-9780199942237
Tian, L., Oviatt, S., Muszynski, M., Chamberlain, B. C., Healey, J. & Sano, A. (2022) Applied Affective Computing. Association for Computing Machinery, New York, United States. https://dl.acm.org/doi/book/10.1145/3502398
Journals:
- IEEE Transactions on Affective Computing https://www.computer.org/csdl/journal/ta
Conference proceedings:
- ACII: Affective Computing and Intelligent Interaction https://dblp.org/db/conf/acii/index
- ICMI: ACM International Conference on Multimodal Interaction https://dblp.org/db/conf/icmi/
- FGR: IEEE Conference on Automatic Face and Gesture Recognition https://dblp.org/db/conf/fgr/