Computer Laboratory

MHHRI Dataset

Multimodal Human-Human-Robot Interactions Dataset for Studying Personality and Engagement

University of Cambridge, United Kingdom
IEEE Transactions on Affective Computing 2017


In this paper we introduce a novel dataset, the Multimodal Human-Human-Robot-Interactions (MHHRI) dataset, with the aim of studying personality simultaneously in human-human interactions (HHI) and human-robot interactions (HRI) and its relationship with engagement. Multimodal data was collected during a controlled interaction study where dyadic interactions between two human participants and triadic interactions between two human participants and a robot took place with interactants asking a set of personal questions to each other. Interactions were recorded using two static and two dynamic cameras as well as two biosensors, and meta-data was collected by having participants to fill in two types of questionnaires, for assessing their own personality traits and their perceived engagement with their partners (self labels) and for assessing personality traits of the other participants partaking in the study (acquaintance labels). As a proof of concept, we present baseline results for personality and engagement classification. Our results show that (i) trends in personality classification performance remain the same with respect to the self and the acquaintance labels across the HHI and HRI settings; (ii) for extroversion, the acquaintance labels yield better results as compared to the self labels; (iii) in general, multi-modality yields better performance for the classification of personality traits.

MHHRI Dataset

The labelled corpus is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License. Please send an e-mail to Hatice Gunes AND Nikhil Churamani confirming your agreement to this and we will send you a password to download the corpus.

Citing our work

If you use any of the resources provided on this page in any of your publications we ask you to cite the following work.

Oya Celiktutan, Efstratios Skordos and Hatice Gunes, Multimodal Human-Human-Robot Interactions (MHHRI) Dataset for Studying Personality and Engagement, IEEE Transactions on Affective Computing, 2017 - DOI 10.1109/TAFFC.2017.2737019.


    You can find here the .tar files of labelled data.