Computer Laboratory

Project suggestions from the Graphics & Interaction Group

Contents

Real-Time Gaze Estimation and Eye Tracking on an Android Portable Device

Originator: Andreas Bulling

Researchers in human-computer interaction as well as hardware manufacturers actively investigate new ways of interacting with portable devices. Latest devices equipped with high-resolution front-facing cameras promise unobtrusive gaze-based interfaces that will become pervasively usable in everyday life. The goal of the current project is to develop and evaluate algorithms for robust gaze estimation and real-time eye movement tracking on an Android portable device. The project builds on a previous project on eye gesture recognition on mobile phones. The project will involve (1) the collection of a large corpus of eye images recorded using a portable device's front-facing camera, (2) the development of low-level image features to estimate discrete gaze directions, and (3) the development and evaluation of algorithms for continuous and real-time eye tracking in a user study.

Requirements: Basic knowledge of computer vision, probabilistic modelling and Android development beneficial
The project might be appropriate for a Part II project, if the student has (some of) the above skills, but is more suitable as an MPhil project.

Multimodal Interaction on an Android Portable Device

Originator: Andreas Bulling

Touch has emerged as the predominant modality for interacting with portable devices, such as mobile phones or tablets. The advent of powerful devices equipped with high-resolution front-facing cameras paves the way for gaze to complement touch and to provide more powerful and natural styles of interaction. The goal of the current project is to develop and evaluate an interface for simultaneous touch- and gaze-based input on an Android portable device. The project builds on a previous project on eye gesture recognition on mobile phones. The project will involve (1) the identification of common tasks performed on portable devices and their adaptation for multimodal interaction, (2) the development of a multimodal interface that implements these tasks, and (3) the development of a realistic experimental scenario and the comparison of multimodal interaction with unimodal eye gesture- and touch-based approaches in a user study.

Requirements: Basic knowledge of Android development, computer vision and interaction design beneficial
The project might be appropriate for a Part II project, if the student has (some of) the above skills, but is more suitable as an MPhil project.

One Day in the Life of the Eyes

Originator: Andreas Bulling

Research on eye-based context recognition has demonstrated that a variety of visual and non-visual activities can be recognised from eye movements. Most work to date has focused on stationary settings and short-term activities typically performed on a desktop computer, such as reading, typing, browsing or watching a video. The goal of the current project is to develop and evaluate algorithms for associating eye movements recorded in mobile daily life settings with longer-term activities or situations. Following an initial data acquisition, the first step will be to analyse the eye movement data and to develop features that are suitable for detecting such activity changes or situations. These features may for example cover basic eye movement characteristics or higher-level behavioural patterns. Afterwards, algorithms for feature extraction and eye movement analysis will be implemented to achieve an entire automatic detection process. The system will be evaluated in a long-term user study.

Requirements: Basic knowledge of signal processing, machine learning and MATLAB/C++ beneficial
The project might be appropriate for a Part II project, if the student has (some of) the above skills, but is more suitable as an MPhil project.

Control room attention models

Originators: Peter Robinson

Many complex industrial systems are run from central control rooms where operators monitor information on multiple screens to identify anomalous conditions. Current design tools for control rooms are limited to 3D models of the hardware which can be used to assess the physical ergonomics, but do not help understand the work of human operators.

This project focuses on developing computational models for predicting the operators' attention so that the human-machine interface could be evaluated and configured properly during control room design. These models are expected to improve arrangement of information shown through the HMI and lessen the operators' risk of missing important information in critical situations. This will involve predicting visual search patterns over an extended display.

Variants on Doo-Sabin subdivision

Originator: Neil Dodgson

Our recent paper, Deriving box-spline subdivision surfaces, describes five dual quadrilateral binary box-spline schemes (Fig. 8) that can all be considered variations of Doo-Sabin subdivision. Two of those five have never been extended to the extraordinary cases (the C2 six-arrow scheme and the C3 eight-arrow scheme). The project is to extend both of these schemes to handle extraordinary cases, then to implement all five schemes and compare them against one another.

Our paper also suggests a couple of other new subdivision schemes, that may be worth investigating as an alternative project. These are described in Sect. 7, bullet points 2 and 3.

This project requires good mathematical knowledge, facility with handling large data structures, and an understanding of 3D graphical presentation and interaction. It might be appropriate for a Part II project, if the student has excellent mathematical skills, but is more suitable as an MPhil project.

Beyond Catmull-Clark subdivision

Originator: Jiří Kosinka

Reimplement (binary) C-C using stencils. This would immediately allow generalising it to non-uniform knots, something we would like in our current EPSRC-funded research. Then, incorporate `averaging' to produce tensor-product stencils so that unequal knot-intervals are supported. Some tuning and tweaking at EVs would be required. Compare results with original Catmull-Clark subdivision, NURSS, DINUS, ... for various averaging techniques (inspired by Hayes). [One could proceed similarly with Doo-Sabin subdivision.]

This project requires good mathematical knowledge, facility with handling large data structures, and an understanding of 3D graphical presentation and interaction. It is suitable as an MPhil project but is too large for a Part II project.

Unlocking NURBS

Originator: Jiří Kosinka

Combine the advantages of NURBS and T-splines (LR B-splines): allow the user to work on a T-mesh, but store the surface as a NURBS surface internally. This does not introduce anything new for approximation or analysis, but could be useful for modelling: existing software with NURBS support would need to be altered only slightly (mostly in terms of GUI), but could keep and make full use of its NURBS library. The user sees only the `free' control points he asks for, the rest remain hidden (but dependent on the free CPs via knot insertion).

This project is an investigation into the interaction mechanisms for handling NURBS. It requires good programming skills for handling the 3D graphics and the 3D interaction. The project is scalable. For a Part II project, it could be a relatively straightforward implementation. For an MPhil project, there are some interesting research questions in addition to the implementation problems.

Driving through Google street view

Originators: Peter Robinson

Google street view presents an environment as a series of still pictures, with an awkward user interface. It would be better if the view changed continuously and could be controlled from a driving simulator.

This first part of the project would be build a rendering system that takes a sequence of images from Google street view and interpolates smoothly between them. This would also involve transforming the images to a consistent frame and handling the images of other vehicles in some way. The second part of the project would be to provide better control, and would involve pre-fetching of images for a variety of paths through the data to give smooth real-time performance.

Facial expression synthesis

Originator: Tadas Baltrušaitis

Facial expressions and head gestures are some of the most important affective (emotional) cues in human interaction. The purpose of the project is to analyse the importance of head pose and facial expression separately and their interaction when expressing emotion/affect. This will aid the design of automatic affect recognition systems and allow for a better understanding of human behaviour. The project will involve (1) tracking facial expressions and head pose (most of the software needed for this is already available through Active Appearance Model and Constrained Local Model libraries), (2) synthesising the tracked expression and pose separately to create new videos (datasets of naturalistic emotions/mental states are available, although the student might opt to record new ones), (3) running a user study on the videos to see the differences in emotion recognition results on different videos. The experiment would be somewhat similar in spirit (although simpler) to the one performed by Boker et al.

Requirements: This project would benefit from previous experience with computer vision. Understanding of statistics would also be helpful. It is suitable as an MPhil project but is too large for a Part II project.

Three-dimensional painting

Originator: Neil Dodgson

We wish to investigate three-dimensional analogues to two-dimensional painting. We envisage a large volumetric "canvas" (say, a 2 metre cube) with the painter manipulating a large stylus to do the painting. One approach is to use voxels (each, say, 1 centimetre cubed) to represent the space; a voxel would have a colour and a transparency. Another approach is to use strokes, where each stroke has a thickness profile and a colour. The challenges are to produce "3D paint" that allows for compelling artistic expression and an interface that can be used by painters other than the project student. The project would use the groups stereoscopic projector, stereoscopic glasses, and a 3D input device, which could be the Vicon system or some alternative.

This is probably a bit too big a project for a Part II student, but could allow an MPhil student to produce an interesting practice-and-experience paper. We may be able to get a practicing artist to engage with the project.

Propose your own project

The Graphics & Interaction Group has a range of interesting hardware. Consider the useful research that could be done if you had access to this and propose something novel and interesting. The equipment currently includes: