Part II and MPhil Projects

Contact: Dr Hatice Gunes -- Hatice.Gunes[@]

Virtual Reality Game-based Cognitive Training for People at Risk of Dementia (MPhil Project)
The management of dementia represents one of the greatest challenges for healthcare systems worldwide. In the absence of cures for diseases such as Alzheimer's disease (AD), there is increasing interest in the prevention of dementia via lifestyle interventions. There is extensive epidemiological evidence to suggest that cognitive, physical and social activity in later life can reduce dementia incidence by enhancing cognitive reserve (CR), such that cognition is maintained in the face of AD pathology. This has stimulated major interest in the use of puzzle- and task-based "brain training" methods as means of preventing AD. However, the efficacy of these approaches have been reduced by low adherence to the training programme, due to reduced user interest in undertaking such tasks long term as a result of limited enjoyment. Furthermore, such tasks engage only cognitive activity and do not promote the physical and social activity that additionally contribute to CR. The problem with adherence has led researchers to explore the idea of "gamification" of cognitive training programmes, on the basis that a games-based approach may be more enjoyable and thus result in improved adherence. This innovative project builds on this concept in three ways. First, it will utilise immersive virtual reality (iVR) platforms for application of game-style cognitive training programmes. It is theorised that iVR-based games are more cognitively stimulating than traditional computer-based games, as a result of the superior sensory involvement, and that their first-person immersive nature may be easier for older people unused to desktop-based computer games. Second, the social activity resulting from future multi-user iVR games would augment the benefits to CR provided by the cognitive activity alone. Finally, our previous work has shown that there are affective, physiological and behavioural differences when people play games in solo, competitive and collaborative modes [1]. These differing responses can feed into adaptive paradigms, such that games can be tailored to individual preferences, creating personalised programmes that may be superior to current "one size fits all" games in terms of adherence and CR enhancement.
In light of the above, this MPhil project will focus on interaction and game design in VR settings for people at risk of AD, and will constitute of the following tasks:

  • Extensive review of existing VR games that can be played in solo, collaborative and competitive modes;
  • Case study with participants to record and measure their nonverbal behaviours and engagement levels when they play existing VR games in the following conditions;
  • (1) Fun games vs. cognitive games aimed at improving memory; (2) Solo vs. collaborative vs. competitive game modes.
  • Qualitative and quantitative analysis of data recorded;
  • (1) Automatic feature extraction from recorded data; (2) Machine learning to train predictive models
  • Based on the findings obtained, designing and implementing a VR game tailored to older people at risk of AD e.g., using Unity 3D software with Unityscript and C# (or similar) for game development. Special attention to be placed on use of system resources (processing and power) as well as on use of lightweight machine learning frameworks for local computation.

  • Suggested reading material:
  • D. Gabana Arellano, L. Tokarchuck and and H. Gunes, Measuring Affective, Physiological and Behavioural Differences in Solo, Competitive and Collaborative Games, Proc. of the International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN 2016), Utrecht, The Netherlands, Jun. 2016.
  • A Discussion of the Use of Virtual Reality in Dementia: al_reality_in_dementia.pdf
  • Virtual Reality app offers unique glimpse into life with dementia:
  • Sending your grandparents to university increases cognitive reserve: The Tasmanian Healthy Brain Project.
  • Does a Combination of Virtual Reality, Neuromodulation and Neuroimaging Provide a Comprehensive Platform for Neurorehabilitation? - A Narrative Review of the Literature.

    Replicating Facial Expressions and Head Movements on Humanoid Robots: Impact on Personality Perception
    Robotic telepresence offers a convenient substitute for face-to-face communication as it provides psychical embodiment and allows the tele-operator to express nonverbal cues such as head movements, hand gestures, facial expressions along with audio cues. In this project, data collected in [1,2] will be used, where the goal was to examine how an individual's personality was perceived when she/he communicated in three different modes: visual-only, audio-only and telepresence. In [1,2], a total of 20 participants were recorded using a RGB camera and only their arm movements were portrayed on a humanoid robot. This project will be focusing on replicating facial gestures and head movements on two different robot platforms: a robot head with a silicon-made actuated skin and a mini humanoid robot with leds on its face. The goal will be to automatically detect and recognise facial gestures and head movements from the recorded videos and develop techniques to replicate the detected/recognised cues on the two robotic heads. In summary, the project constitutes of the following tasks:

  • Developing a novel technique for recognising facial action units and filtering over time to display on a robotic head platform.
  • Collecting annotations of personality for the generated robotic head/face displays
  • Comparing the personality annotations in order to model personality perception across different robotic platforms

  • Suggested reading material:
    [1] Celiktutan et al., Personality Classification from Robot-Mediated Cues, RO-MAN'16.
    [2] Bremner et al., Personallity Perception of Robot Avatar Teleoperators, HRI'16.

    Personality synthesis in humanoid robots
    Personality is an important feature in interpersonal communication, utilised in judgements of a conversational partner and aiding behaviour predictions. Hence synthesis of personality would aid the interaction capabilities of socially interactive robots. Recent work on automatic personality analysis has revealed that cues for personality are present in non-verbal communications such as hand gestures. This project aims to look at how such cues might be expressed on a humanoid robot (NAO), and whether synthesised personalities can be recognised. It will achieve the following:

  • Identification of key personality cues from automatic analysis data
  • Parameterisation of identified personality cues so they can be automatically applied to robot communication
  • Synthesis of several personalities applied to a scripted multi-modal communication on the NAO robot
  • Experimental design and analysis of robot personality recognition

  • Suggested reading material:
    Paul Bremner, Oya Celiktutan, Hatice Gunes: Personality Perception of Robot Avatar Tele-operators. HRI 2016: 141-148

    Mapping of tele-operator behaviour to idealised motion
    Using humanoid robots for tele-presence has been shown to offer a number of advantages over more traditional screen based approaches. Thus far we have used motion capture based control, replicating the operator's motions on the robot. However, this often results in motion that is not best suited to performance by the robot. The aim of this project is to develop a better tele-presence control method by using machine learning to produce a system that maps operator behaviours to idealised motion suited to the robot. It will achieve the following:

  • Investigate idealised motion from animation/puppeteering/human behaviour analysis. Conduct empirical studies on how this applies to the NAO robot.
  • Determine an appropriate machine learning approach, and design a suitable system.
  • Produce a set of training data by recording human motion and scripting appropriate idealised motions
  • Experimental design and analysis of system performance

  • Suggested reading material:
    O. Celiktutan, P. Bremner and H. Gunes, Personality Classification from Robot-mediated Communication Cues, Proc. of IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Aug. 2016.

    Egocentric Body Movement Analysis based on Visual and Music Signals in a Party Scenario
    In a party scenario, people tend to synchronise with music and engage in dancing. Understanding attendees' movements is important in measuring crowd's satisfaction, and this information can be used to control the music selection process [1]. This project aims at analysing the relationship between body movements of party attendee' and music played in such a setting. Data was collected using egocentric centric cameras worn by six party attendees on their chests, for a total of 45 minutes. The project constitutes of the following four tasks:

  • Manually segmenting egocentric recordings into short, semantically meaningful clips.
  • Feature extraction from egocentric visual data to capture body movements.
  • Feature extraction from audio/music signals using MIR toolbox [4].
  • Developing an unsupervised technique for discovering relationship between visual features and music features.

  • [1] Kuhn et al., Sensing Dance Engagement for Collaborative Music Control, ISWC'11.
    [2] Poleg et al., Temporal Segmentation of Egocentric Videos, CVPR'14.
    [3] Ryoo et al., Pooled Motion Features for First-person Videos, CVPR'15.

    Detecting Activities and Discovering Events in Multi-person RGB-D Videos
    This project focuses on activity recognition and multi-person event detection in RGB-D videos captured in a party scenario. Multi-person event detection is a challenging task due to low resolution of faces and severe occlusions. Data consists of approximately 50 participants, and was collected by two top-view Kinect depth sensors for a duration of 45 minutes. The goal of this project will be developing novel computer vision and machine learning methods:

  • To detect upper bodies and estimate upper body orientations in RGB-D sequences.
  • To recognise human activities such as walking, standing and dancing.
  • To detect socially related people based on their spatial arrangements and shared focus of attention.
  • To discover events such as approaching, forming a group, dancing in pairs, etc.
  • In order to validate the developed methods, the dataset will need to be annotated manually using an annotation tool (that is readily available) with respect to people and events, in a frame-by-frame basis.