Research

Current projects

My current research is focussed on the general area of distributed systems, focusing specifically on multi-scale computation and storage over heterogeneous resources. This deliberately vague tagline can mean a few things:

Data centres and cluster computing

I am currently working on two systems for parallel data-flow computing at scale and over heterogeneous resources:

• Firmament is a distributed, task-parallel cluster manager for data-flow jobs. It is based on a master-less design and supports novel resource-aware scheduling abstractions and algorithms.
• DIOS, the Data-Intensive Operating System, is a distributed operating system for warehouse-scale computers and data centres. It provides an execution environment and system services for data-flow jobs, and works in concert with Firmament to run a diverse set of data-flow jobs over heterogeneous collections of resources.

I am also involved in the R2D2 project on bounded-latency data centre networking: messaging in DIOS is an application that benefits from R2D2s unbuffered, explicitly scheduled network communication.

IPC performance on multi-core

As part of my work on multi-scale computation and scheduling strategies for collections of resources with heterogeneous features and communication channels, I became interested in the performance of local message passing schemes on a machine (usually under the umbrella term of "inter-process communication", IPC, in an operating system). I helped design and implement the ipc-bench benchmark for IPC performance, and set up the results database on the web, showing clearly the signifcant variance in IPC performance across different micro-architectures and communication mechanisms.

As a follow-up to our findings with ipc-bench, we are now working on a unified IPC library called Fable, which automatically choses the optimal transport between different endpoints, and reconfigures automatically if conditions change.

» Help us by running ipc-bench on your machines!
» Our position paper introducing Fable

Ciel

I actively participate in the Ciel project of the NetOS group. Ciel is an execution engine for distributed, parallel computation. Like previous frameworks (such as MapReduce and Dryad), it masks the complexity of distributed programming. Unlike those frameworks, it offers a full general-purpose programming environment for expressing distributed coordination, which enables developers to implement algorithms using arbitrary task structures in many languages, including Java, C/C++, Python, Scala, OCaml and the bespoke Skywriting scripting language.
A paper on Ciel was published in NSDI 2011, and we have also produced a draft paper on Ciel's language support and advanced features. I have myself done some work on running Ciel within the domain of a single many-core machine, and published a short paper in SFMA 2011.

» Find out more!
» Our NSDI paper
» Draft paper on language support and advanced features
» My paper on running Ciel on many-core

Investigating personal storage

In the past, I have also taken in interest in ubiquitous storage and systems supporting variable trade-offs for diverse storage requirements, both on the infrastructure side and in ways of developing ubiquitous access.

In 2010/2011, I have conducated a survey about personal data storage. The aim of this work was to find out how people store their data and how the "average" person's data is structured. To investigate this, I ran ran a web-based survey combined with asking participants to download an automated tool that scans their personal data and gathers some fully anonymized data, deleting itself afterwards.

» Find out more and participate!

I am still continously analysing the data received from participants, and will use it to investigate novel storage solutions for federated personal data.

Nimbus

Nimbus is the prototype of an intelligent storage manager for federated personal data. Informed by results from the personal data survey (see above), I am working on an architecture that better supports the requirements of modern data storage patterns than traditional storage media or online services with doubtful privacy ethics.

To find out more about Nimbus, have a look at two posters about it.

Past projects

Proteus

My undergraduate dissertation was in the area of Computer Vision, focussing on interactive 3D structure-from-motion. Proteus is a versatile, interactive 3D structure-from-motion implementation that allows a user to trace the contours of an object in a video and then attempts to extract a 3D model from it.
More information about it can be found on the Proteus project page.

This work was supervised by Christian Richardt.