Grants & Projects
Innovate UK Sensing Feeling Project. Funded by the Technology Strategy Board (TSB), 2016-2018 (~£500k).
EPSRC HARPS Project - Humans and Robots in Public Space. Funded by the Engineering and Physical Sciences Research Council (EPSRC), 2013-2017 (£2M).
EPSRC MAPTRAITS Project. Funded by the Engineering and Physical Sciences Research Council (EPSRC), 2013-2014 (£123k).
Affective Computing for Mobile HCI, with Hazim K. Ekenel and Istanbul Technical University, funded by the British Council under the UK-Turkey HE Partnership Programme, 2012-2014 (£25k).
Joint Small Equipment Grant, with QMUL EECS Early Career Researchers, funded by the Engineering and Physical Sciences Research Council (EPSRC) (£500k).
Research Interests
affective computing
social signal processing
human-robot interaction
social robotics
human behaviour understanding
multimodal interaction
signal processing
computer vision
applied machine learning
computer mediated collaborative and creative interactions
aesthetic canons: representation, automatic analysis and prediction.
Past Research
Postdoctoral Research, Imperial College London, Intelligent Behaviour Understanding Group (iBUG) (2008-2011)
At Imperial College, I worked in Maja Pantic 's iBUG research group on the European Community (EU FP7) award winning project SEMAINE that aimed to build a multimodal dialogue system which can interact with humans via a virtual character and react appropriately to the user's non-verbal behaviour, and MAHNOB that aimed at multimodal analysis of human naturalistic nonverbal behaviour.
The first demonstration of Semaine, titled 'A Demonstration of Audiovisual Sensitive Artificial Listeners', won best demo award at ACII 2009, the International Conference on Affective Computing and Intelligent Interaction, Amsterdam, the Netherlands, in September 2009.
We also demonstrated the Semaine system at IEEE International Conf. on Automatic Face and Gesture Recognition in March 2011. Here is a representative video of the interaction taking place between a human user and one of the virtual characters, recorded with the latest system.
Ph.D. Research, University of Technology, Sydney (UTS), Faculty of Information Technology, Computer Vision Research Group (CVRG) (2003-2007)
My PhD research focused on building a multi-modal/cue module that can extract features from expressive face and upper-body gestures (e.g., shoulder movements and hand gestures) using computer vision and image processing techniques and integrate these features using machine learning and pattern recognition techniques. During my PhD research I explored novel research grounds such as multi-modal/cue affective database creation from multiple sensors and annotation of such data (differences in interpretation and labelling: face vs. face-and-body), detection of temporal phases (neutral, onset, apex and offset) decoupling temporal phases from the spatial features in order to achieve higher accuracy in affective state recognition, synchronization of the bimodal data to the purpose of multimodal fusion, and obtaining optimal fusion by experimenting with various strategies. At the time of its completion, it was one of the first attempts reported in the literature to recognise human affect like happiness and sadness from multiple visual cues such as face and body gestures. Another major contribution of my PhD work was the FABO database containing labelled videos of posed face-and-body affective displays. This database has been made publicly available for research purposes. This research was conducted under the supervision of Prof. Massimo Piccardi.
Visiting Ph.D. Student, Delft University of Technology, Faculty of Electrical Engineering, Mathematics and Computer Sciences, Man-Machine Interaction Group, Delft, The Netherlands (Nov. 2005- Feb. 2006)
The focus of my visit was on finding how humans use their face and body when they are communicating affectively in natural settings and how their face and body displays differ when they are posing upon request for affective data acquisition. Thus, the first aim of this research was to obtain visual spatiotemporal analysis of posed versus spontaneous facial and bodily expression by exploring data acquired both in laboratory and (more) natural settings. The next step in this research was to automatically detect, recognize and compare these cues in a multi- cue/modal manner and conclude whether the observed differences were sufficient for a machine to learn to distinguish between spontaneous vs. posed affective displays. Overall, it was possible to distinguish between posed and spontaneous smiles by fusing information coming from head, face, and shoulder channels at different levels of abstraction (i.e., early, mid-level, and late fusion). The results obtained contributed to the field of Automatic Analysis of Human Spontaneous Behaviour that aims to develop natural human machine interfaces by analysing the affective state of the users. The research was conducted under the supervision of A/Prof. Maja Pantic.
Research Assistant, University of Technology, Sydney (UTS), Faculty of Information Technology, Computer Vision Research Group (CVRG) (Feb. 2003- Feb. 2004)
My research during this period tackled the problem of automatic assessment of aesthetics. In the first part of this research, we evaluated the extent of universality of aesthetics by asking a diversified set of human referees to grade a collection of female facial images in terms of their facial aesthetics. Results obtained showed that the different individuals generally provided unimodal and compact grade histograms, thus well supporting the concept that perception of aesthetics is universal to a certain degree. Later, we introduced an approach to automatically measure aesthetics based on automated extraction of facial features and supervised classification. We presented an efficient procedure for automatically measuring facial features from face images by means of image analysis operators. For supervised classification, we used such extracted facial features and the average human grades from a set of images to train an automated classifier. The accuracy achieved on an independent test set and from cross-validation proved that the classifier can be effectively used as an automated tool to reproduce an average human judgement on facial aesthetics.