Computer Laboratory

Project suggestions from the Graphics & Interaction Group

Training conductors

Originator: Mark Gotham and Neil Dodgson

Gotham is a PhD student in the Faculty of Music, By-Fellow and Director of Music at Churchull College. Research Context: The conductor's art has traditionally been a subject of mystery, but their technique has recently become a lively area of technical research. It is a quintessential example of an emerging field of systematic study which has been made possible by the development of modern technologies; through the Vicon system of motion detection and other means, it is now possible to capture much of the conductor's motion in three dimensions, including the patterns of acceleration as well as the shapes they trace. Among the most interesting early results to emerge from this area, Woellner et al. (2012) showed that quantitatively averaged motion is easier to synchronise with than any individual conductor's pattern: that is, the 'average' conductor is the clearest for musicians to follow. Goals: If prototypical forms do indeed exist and are the easiest to follow, then they provide a clear model for all seeking to learn or improve their basic conducting technique; yet no attempt has yet been made to create a medium for enabling this. This project seeks to develop new means for analysing and teaching basic technique by comparing users' motion with prototypical forms, thus guiding them on an individual basis towards self-improvement. This promises to be much more efficient and inclusive than the current situation in which conductors learn their basic technique from some combination of books (very limited in conveying motion); individual lessons (costly and exclusive); and 'on the job' practice (wasteful of musicians' time and goodwill). Student Project: The project is to develop and assess the first, rudimentary forms of this interface. The project might be scoped to include some of the following: (1) measurement of users' acceleration patterns (without the requirement to register position); (2) motion capture of a user's movements; (3) video capture of a user's movements. Captured movements might be (4) overlayed on an animated display of prototypical motion (for self-guided improvement) or (5) systematically compared with the prototypical motion for structured feedback. The project will require some facility with special hardware and its associated software. Possible hardware includes basic accelerometers (such as those in smart phones), motion capture systems such as the Kinect, Leap Motion, or Vicon, and the possible integration of multiple devices. The group has both Kinect and Vicon available. Personnel: Gotham, a conductor and mathematically-inclined music theorist, will be able to provide supervision on the application domain, but the student will need to find a computer science supervisor to formally supervise the project and to handle the technical side of the work.

Visualising a Dynamic Global Vegetation Model

Originator: Tim Tito Rademacher and Neil Dodgson

Rademacher is a PhD student in the Department of Geography. This project is to produce an application to visualise and map outputs from a Dynamic Global Vegetation Model (Hybrid IX). A Dynamic Global Vegetation Model (DGVM) is used to understand feedbacks between soils, the atmosphere and plants on a planetary scale. Hybrid IX is a state-of-the-art DGVM developed by Dr Andrew Friend in the Department of Geography, which is undergoing continuous development. Hybrid IX models the development of terrestrial vegetation worldwide from factors as diverse as photosynthetic processes in needle-leaved boreal trees and the effect of drought mortality on Amazonian rain forests. Hybrid IX is designed with the aim of dramatically improving our understanding of the impact of climate on terrestrial vegetation and vice versa. Understanding different model formulations or comparing different models is often a visual exercise due to the enormous number of outputs generated and the various scales involved, ranging from a single plant to the whole globe. The team at the Department of Geography is looking for an enthusiastic undergraduate (no ecological knowledge required!) to create a tool that facilitates the step of visualising and/or mapping model outputs. The aim of this project is to produce an applications that can read in Hybrid IX outputs and produce visuals of these. Especially, a better representation of modelled trees and other plant shapes in several spatial and temporal dimensions is desired. The resulting visuals would be used to gain a better understanding of the model, for further model development, to compare the model to observations, to communicate results and as a teaching tool. Rademacher can provide supervision on the modelling tool and on the types of visual output that would be useful to geographers. An interested student will need to find a second supervisor who can provide supervision on programming.

Live coded video processing architecture

Originator: Alan Blackwell with Sam Aaron

Sam Aaron's Sonic Pi language for live-coded music is a local product that has achieved huge popularity and media coverage. Funded by the Raspberry Pi Foundation, it is also availabnle for Mac and PC platforms.

The goal of this project is to create an architecture for interacting with multiple video streams, modifiable in real time through live-coding styles of software development. The starting point will be modelled on the Supercollider architecture for real-time processing of audio and event streams, but extended to support image and video frame data. It may not be necessary to use the Supercollider source code, although this will provide a source of guidance. The implementation language is likely to be C++.

Evaluation can be done in the Lab, based on video throughput, time resolution, latency and jitter from filter operations. As an extension, it would also be possible to package "Video Pi" (or choose your own name if you like) into the Sonic Pi editing environment so that kids can create their own video mixers or filters connected to an external camera. There will also be opportunities for future funded research in this area.


Originator: Peter Robinson

Touch screens driven by Raspberry Pi computers are being installed outside each office on the SS corridor. These are being configured initially to show calendars and to pass messages to and from people who are working away from the laboratory. However, there are many other possibilities. Think of one and implement it!

Intelligent music player

Originator: Vaiva Imbrasaitė

As the sizes of our digital music libraries are growing, creating and editing our music playlist is becoming more and more painful. The simplest solution is to listen to a shuffle playlist of the whole music library, but that will inevitably lead to unwelcome changes in the music flow, and a large number of skipped songs. The idea of this project is to create an intelligent playlist generator. A number of different heuristics as well as machine learning techniques can be used. The playlist generator can also be based on recent research articles. The project should also include a user study to evaluate the performance of the music player.

This can be an undergraduate or masters project, depending on the complexity of the playlist generator and the extent of the user study.

Image-based rendering from multi-sensor fusion

Originator: Dongjoe Shin

Image-Based Rendering (IBR) is an approach to render a novel view from a set of images taken at different positions without knowing any 3D geometry of a scene. There are many different approaches to realise IBR, but in this project a student will explore a simple approach using view interpolation (see link ). Since the view interpolation basically creates a virtual view by linearly interpolating both positions and colours of corresponding points between views, developing efficient algorithm to estimate a complete optical flow would be one of the main challenges for a real-time application of this approach.

In this project, a student will explore a possibility of using a 3D sensor for fast estimation of dense optical flow of high resolution images. Since the resolution of a depth map from Kinect is generally lower than that of a digital image and a resulting depth map is prone to contain noise including large holes in a depth map. This project will require to develop a robust inter and extra-polation algorithm after registering coarse 3D depth data to each image from an arbitrary position.

This project is suitable for Part II students who have basic knowledge of image processing and stereo matching, but can be extended for MPhil students.

Novel interactive data visualisations

Originators: Alan Blackwell with Advait Sarkar and Ben Azvine (BT research)

Large networks of the kind operated by BT are a source of huge quantities of time-varying data with many variables. A wealth of information can be extracted from such data, but initial exploration of the dataset may be formidable, particularly when the features of the dataset are unknown. There are a few standard means of data visualisation including trend graphs, bubble diagrams, network diagrams, pie charts, geographical maps, sun ray diagrams, and radial views. However, these represent a relatively limited range of statistical methods. The goal of this project is to build on the capabilities of statistical analysis packages such as R and NumPy, to create new visualisations as an intuitive alternative to existing statistics packages and open source visualisation tools.

Musical interfaces to programming languages

Originators: Alan Blackwell and Sam Aaron

Live Coding is an exciting music genre featured not only in experimental art music, but in jazz improvisation and Algoraves (featured in Wired Magazine in August). Cambridge is one of the international centres of Live Coding research and performance. The goal of this project will be to integrate some newly acquired musical equipment (including a MIDI drumkit) into the programming environments used in improvised programming.

Control room attention models

Originators: Peter Robinson

Many complex industrial systems are run from central control rooms where operators monitor information on multiple screens to identify anomalous conditions. Current design tools for control rooms are limited to 3D models of the hardware which can be used to assess the physical ergonomics, but do not help understand the work of human operators.

This project focuses on developing computational models for predicting the operators' attention so that the human-machine interface could be evaluated and configured properly during control room design. These models are expected to improve arrangement of information shown through the HMI and lessen the operators' risk of missing important information in critical situations. This will involve predicting visual search patterns over an extended display.

Beyond Catmull-Clark subdivision

Originator: Jiří Kosinka

Catmull-Clark is a recursive subdivision scheme that produces smooth curved surfaces from a polyhedral mesh. At each subdivision step, simple rules are used to create a finer mesh. The process terminates when polygons are considered small enough (about pixel-sized in graphics applications).

In this project, we re-implement Catmull-Clark subdivision using stencils to create the vertices of the new mesh. This would immediately allow generalising it to non-uniform knots, something we would like in our current EPSRC-funded research. Then, incorporate `averaging' to produce tensor-product stencils so that unequal knot-intervals are supported. Some tuning and tweaking at EVs would be required. Compare results with original Catmull-Clark subdivision and other subdivision surfaces. [One could proceed similarly with Doo-Sabin subdivision.]

This project requires good mathematical knowledge, facility with handling large data structures, and an understanding of 3D graphical presentation and interaction. It is suitable as an MPhil project but is too large for a Part II project.

Games on subdivision surfaces

Originator: Jiří Kosinka

Subdivision surfaces based on quadrilateral meshes (e.g. the popular Catmull-Clark scheme) provide a natural extension of regular rectangular grids to surfaces of arbitrary topology. The differences are that other than four quadrilaterals may meet at a vertex and that the mesh may form a close surface. The project would be to generalise and implement games such as snake, the game of life, minesweeper, or any suitable game of your choice, to quad meshes generated by subdivision.

The project is suitable as a Part II project and can be extended to an MPhil project (e.g., by exploiting the multiresolution nature of subdivision, designing smooth camera motions or adding multiplayer support).

Splines with semi-sharp features

Originator: Jiří Kosinka

Splines are usually designed using control points. The idea behind this project is to generalise this concept to support control vectors as well. This approach, if implemented using subdivision, would allow intuitive modelling with semi-sharp features.

This project is scalable (implementation can support either curves only, or both curves and surfaces) and thus it is suitable as a Part II or an MPhil project.

Hayes surfaces via subdivision

Originator: Jiří Kosinka

The original idea of Hayes was to generalise B-spline surfaces by allowing knots to be functions of the parameter. The project is to reimplement and extend this idea using subdivision. Compared to standard B-spline surfaces, this would give rise to a more flexible modelling tool within a multi-resolution framework.

This project might be appropriate for a Part II project, if the student has excellent mathematical skills, but is more suitable as an MPhil project.

Dramatic decimation for curves

Originator: Jiří Kosinka

Our current EPSRC-funded research focuses on dramatic decimation for surfaces: given a dense triangular mesh, extract a good sparse control mesh that captures the main features of the input. Applications range from multi-resolution modelling to data compression.

However, for open meshes, the boundary needs to be treated separately. The project is: given a curve (dense polygon), find a good sparse representation for it. One can look at singularities, inflections, other special points, or use Morse theory (with curvature). B-spline subdivision (possible with features) can then be used to reconstruct the input curve.

This project is suitable for a Part II project or an MPhil project.

Driving through Google street view

Originators: Peter Robinson

Google street view presents an environment as a series of still pictures, with an awkward user interface. It would be better if the view changed continuously and could be controlled from a driving simulator.

This first part of the project would be build a rendering system that takes a sequence of images from Google street view and interpolates smoothly between them. This would also involve transforming the images to a consistent frame and handling the images of other vehicles in some way. The second part of the project would be to provide better control, and would involve pre-fetching of images for a variety of paths through the data to give smooth real-time performance.

The big question would be to decide between a model-based approach and an image-based approach. On the one hand, extracting a 3D model from the Street View images would be hard, but it would allow smooth animation while driving. That would require a method for removing inconsistent clutter like people and other vehicles. On the other hand, smooth interpolation between discrete images seems like a hard problem.

Requirements: This project would benefit from previous experience with computer vision. It would make a hard MPhil project and is too large for a Part II project.

Facial expression synthesis

Originator: Tadas Baltrušaitis

Facial expressions and head gestures are some of the most important affective (emotional) cues in human interaction. The purpose of the project is to analyse the importance of head pose and facial expression separately and their interaction when expressing emotion/affect. This will aid the design of automatic affect recognition systems and allow for a better understanding of human behaviour. The project will involve (1) tracking facial expressions and head pose (most of the software needed for this is already available through Active Appearance Model and Constrained Local Model libraries), (2) synthesising the tracked expression and pose separately to create new videos (datasets of naturalistic emotions/mental states are available, although the student might opt to record new ones), (3) running a user study on the videos to see the differences in emotion recognition results on different videos. The experiment would be somewhat similar in spirit (although simpler) to the one performed by Boker et al.

Requirements: This project would benefit from previous experience with computer vision. Understanding of statistics would also be helpful. It is suitable as an MPhil project but is too large for a Part II project.

Visual services

Originator: Peter Robinson

Tangible user interfaces could be built using a standard mobile phone and 2D bar-codes [Controlled availability of pervasive Web services, IEEE ICDCS 2003]. Design, build and evaluate such a system.

Propose your own project

The Graphics & Interaction Group has a range of interesting hardware. Consider the useful research that could be done if you had access to this and propose something novel and interesting.