MPhil and Part II projects 2012/2013

Physical Control of a Programming Editor for Live Coding

Monome Drum Kit

Although a wide range of interaction peripherals are now available for workstations, most programming editors stick with the keyboard and mouse. Do these standard peripherals offer the most effective interface to programming or are they just a surviving relic? What might be the requirements of a programming environment for more esoteric physical interfaces and how might we measure their efficacy?
This project will explore the opportunities alternative interfaces offer to the programming workflow. As the context for this exploration we shall consider "live coding" where programmers construct and manipulate code on-the-fly for artistic means such as making music: Tools for live coding a musical performance provide a novel set of constraints on programming workflow where live, highly reactive and feedback-oriented interfaces are an important requirement.
Suggested work programme
The technical starting point will be to expand the typical REPL-oriented approach to performing music with live coding toolkits such as Overtone. This would require obtaining a basic understanding of the Clojure programming language and live musical performance with algorithmic synthesisers. One possible approach would be to extend the programmer's editor Emacs with a range of new syntax-directed editing interfaces controlled by external devices such as rotary controllers, monomes, MIDI surfaces or instruments.
As an extension, it would be interesting to explore how this new approach can be directly included in performance workflows such that new interfaces may be designed and created at runtime with little or no interference to the ongoing performance. It would also be interesting to explore how these interface techniques might be applicable to more traditional programming contexts.
Contact:, or for MPhil

Functional Representation of Time for Control Applications

Conveyor Ladder Diagram

Clojure is a new functional language that executes on the JVM, and has recently come into high demand as a professional implementation platform.
New research by Sam Aaron and Jeff Rose has created a Clojure extension for real-time applications called Overtone (see
Although originally developed for time-based description of musical structure, Overtone includes explicit models of time that potentially make it well-suited to industrial real-time control applications.
The goal of this project is to create a graphical front-end to Overtone programs, for example in a format related to the ladder logic diagrams that are traditionally used by engineers to program factory automation systems. Ladder diagrams are poor in their representation of time (for example, they can suffer from race conditions), meaning that an Overtone/Clojure back end may be a good choice to express and debug execution and timing constraints.
Suggested work programme
The first step will be to learn Clojure, and create an Overtone simulation of the mechanical behaviour of a simple assembly line - for example a rotating stamp, short conveyor belt and packing robot. Timing and mechanical variability will be expressed in Overtone, resulting in a time sequence of simulated sensor outputs to be used by an automated controller.
The next step will be to create a graphical interface in ladder logic form, that can be edited by the user to define responses to the sensor outputs. This interface can be created in Java, with a Clojure back end that generates Clojure code equivalent to the ladder logic.
Finally, the ladder logic controller will be connected to the assembly line simulation, in order to test the ladder logic program under various conditions of machine performance and simulated faults.
As an extension, the simulated assembly line can also be presented to the user as a Java animation, so that the performance of the controller can be viewed as a live animation. In principle, the assembly line could also be configured via a graphical interface, although this would most likely be beyond the scope of a Part II project.
Take a small number of test cases based on specifications of actual machines, and demonstrate that these can be simulated, and the real- time simulation correctly controlled, using the system. Conduct simulated experiments in which the relative timing of machine components is varied, with randomly injected fault conditions corresponding to component failure.
The success criterion is for the control logic should operate as expected within a defined safety/performance envelope, over a range of simulation conditions. The evaluation phase will collect simulation data in order to demonstrate that this goal has been reached.
Design of evaluation scenarios will involve some discussion with one or more engineers from a specialist industrial control company such as Quin Systems or TAP Biosystems. An optional additional evaluation exercise could evaluate the relative usability of Clojure and ladder logic with respect to other automation languages used by companies like these.
Contact:, or for MPhil

Video Processing Language for the Raspberry Pi

Raspberry Pi Kodu

The Raspberry Pi architecture essentially provides a set-top box with an end-user programmable architecture. However, most users of the device are not yet exploiting its video processing capabilities. In part, this is because the programming APIs for video codecs under Linux are not accessible via educational programming languages like Scratch.
The objective of this project is to extend an existing visual programming language with syntax to support live video processing. The general syntactic style could be based on systems such as Microsoft Research Kodu (pictured), although several alternatives are also available within the Computer Laboratory.
Suggested work programme
First step will be to experiment with the existing API to video codecs used on the Raspberry Pi, and evaluate the performance of the GPU under live transformations such as image scaling and warping, streaming from online video sources, processing of input from a camera module, and compositing of video data with 2d or 3d graphic renderings. Based on this evaluation, a selection of end-user programmable functions will be designed that are suitable for the Raspberry Pi graphics architecture.
A visual language syntax will then be selected, derived from one of a number of data flow, functional and constraint languages that have been used for similar signal processing and image processing tasks in the past.
A minimal visual syntax will be defined. In order to keep the scope of the project manageable, it is important that the initial syntax be sufficient only to support the programmable functions and their parameters, with sufficient expressiveness to modify and bind alternative values under user-program control (possibly extended via external I/O to allow users to create novel physical controllers).
A basic editor for this visual syntax will be implemented using the graphics facilities on the Raspberry Pi. Ideally, this should allow dynamic drawing and drag and drop of syntax components, but if necessary can be keyboard controlled. Dynamic composition of programs can be considered as an extension in the project plan.
A more ambitious extension would be to enhance the language syntax with more general purpose compute functions, for example sufficient to express a simple algorithm animation such as Quicksort using basic video blocks, or create turtle-graphics style geometry.
A test suite will be defined for the language, representing a range of end-user video-processing scenarios (mashups, live VJ performance, creation of titles and fades etc). This will be validated by interviews with a small number of sample users. The main success criterion will be the successful creation of sample programs in the new language that demonstrate the video processing behaviours in the test suite.
As an extension, a user study could be carried out with a small sample of typical users, comparing the usability of the new language to others in this class.

Click to return to Alan Blackwell's home page.