Computer Laboratory > Teaching > Course material 2009–10 > Innovative user interfaces


Innovative user interfaces

Reading group for the MPhil in Advanced Computer Science.
Peter Robinson.
Michaelmas Term, Tuesday, 16:00-18:00, SS03.


This module aims to provide a review of innovative user interfaces.

On completing the module, students should understand the development of user interfaces throughout the history of computing, and be aware of recent research directions.


The module consists of eight two-hour meetings of a reading group.

Each participant will be expected to read some relevant papers about each topic, and write an essay on it to be submitted before the meeting. Each participant will also be expected to give a presentation about one of the weekly topics to the other members of the class.

The final meeting at the beginning of the Lent Term will give an opportunity for each participant to make a further presentation about a recent paper of their own choosing.


It is assumed that students will have taken an undergraduate course in human-computer interaction and will be familiar with the standard literature such as:

Interactive system design
William Newman & Mik Lamming.
Addison-Wesley, 1995.

Preliminary reading

Students intending to take the course should read the following before the beginning of term:

The psychology of everyday things
Donald Norman.
Basic Books, 1988.
The computer for the 21st century
Mark Weiser.
Scientific American, September 1991.
Reprinted in Mobile Computing and Communications Review, July 1999.
The intuitive beauty of computer-human interaction
Communications of the ACM 43(3), March 2000.


Week 1: Early inspiration

Participants do not need to submit an essay in advance of the first meeting, but should still read the following papers and come prepared to discuss them.

As we may think
Vannevar Bush.
Atlantic Monthly, July 1945.
Man-computer symbiosis
Joseph Licklider.
IRE Transactions on Human Factors in Electronics, March 1960.
Ivan Sutherland.
AFIPS Spring Joint Computer Conference, 1963.
Full dissertation reprinted as CL Technical report, September 2003.
Videos 1/2 and 2/2.
A methodology for user interface design
Charles Irby, Linda Bergsteinsson, Thomas Moran, William Newman & Larry Tesler.
Xerox PARC, 1977.

Week 2: Windows, icons, mice and pointing

The model human processor
Stuart Card, Tom Moran & Alan Newell.
Handbook of Perception and Human Performance, Wiley, 1986.
Star graphics: an object-oriented implementation
Daniel Lipkie, Steven Evans, John Newlin & Robert Weissman.
ACM SIGGraph 16(3), July 1982.
Videos 1/2 and 2/2.
The X Window System
Robert Scheifler & Jim Gettys.
ACM Transactions on Graphics 5(2), April 1986.

Week 3: Video user interfaces

Interacting with paper on the DigitalDesk
Pierre Wellner.
Communication of the ACM 36(7), July 1993.
BrightBoard: A video-augmented environment
Quentin Stafford-Fraser & Peter Robinson.
ACM Human Factors in Computing Systems, April 1996.
High-resolution interactive displays
Mark Ashdown, Philip Tuddenham & Peter Robinson.
Tabletops — Horizontal Interactive Displays, Springer, 2010.

Week 4: Direct manipulation

Direct manipulation: a step beyond programming languages
Ben Shneiderman.
IEEE Computer 16(8), August 1983.
Toolglass and magic lenses: the see-through interface
Eric Bier, Maureen Stone, Ken Pier, William Buxton & Tony DeRose.
ACM SIGGraph, August 1993
Tangible bits: beyond pixels
Hiroshii Ishii.
ACM Conference on Tangible and Embedded Interaction, February 2008.

Week 5: The disappearing computer

Some Computer Science issues in ubiquitous computing
Mark Weiser.
Communications of the ACM 36(7), July 1993.
Sentient computing
Andy Hopper.
Philosophical Transactions of the Royal Society A 358, August 2000.
Building disappearing computers
Daniel Russel, Norbert Streitz & Terry Winograd.
Communications of the ACM 48(3), March 2005.

Week 6: Special purposes

What's real about virtual reality?
Frederick Brooks.
IEEE Computer Graphics and Applications 19(6), November 1999
Bridging the physical and digital in pervasive computing
Steve Benford, Carsten Magerkurth & Peter Ljungstrand.
Communications of the ACM 48(3), March 2005.
Pragmatic research issues confronting HCI practitioners when designing for universal access
Simeon Keates.
Universal Access in the Information Society 5(3), November 2006.

Week 7: Affective computing

Facial Expression and Emotion
Paul Ekman.
American Psychologist 48(3), April 1993.
Affective Computing
Rosalind Picard.
MIT Media Lab, 1995.
Computers that care
Scott Brave, Clifford Nass & Kevin Hutchinson.
International Journal of Human-Computer Studies 62, January 2005.

Week 8: Emotional inference

Real-time inference of complex mental states from facial expressions and head gestures
Rana el Kaliouby & Peter Robinson.
Real-time vision for HCI, Springer-Verlag, 2005.
Classification of complex information: Inference of co-occurring affective states from their expressions in speech
Tal Sobol-Shikler & Peter Robinson.
IEEE Transactions on Pattern Analysis and Machine Intelligence 31, May 2009.
Detecting affect from non-stylised body motions
Daniel Bernhardt & Peter Robinson.
International Conference on Affective Computing and Intelligent Interaction, September 2007.

Individual presentations

Back-of-device interaction allows creating very small touch devices
Gerry Chu & Patrick Baudisch
ACM Human Factors in Computing Systems, April 2009.
Presented by Alexander Kuscher
SenseableRays: Opto-haptic substitution for touch-enhanced interactive spaces
Jun Rekimoto
ACM Human Factors in Computing Systems, April 2009.
Presented by Melinos Averkiou
Interactivity attributes: A new way of thinking and describing interactivity
Youn-kyung Lim, Sang-Su Lee & Kwang-young Lee
ACM Human Factors in Computing Systems, April 2009.
Presented by Zeynep Sagar
Natural throw and tilt interaction between mobile phones and distant displays
Raimund Dachselt & Robert Buchholz
ACM Human Factors in Computing Systems, April 2009.
Presented by Ovidiu Popa
Wearable EOG goggles: eye-based interaction in everyday environments
Andreas Bulling, Daniel Roggen & Gerhard Tröster
ACM Human Factors in Computing Systems, April 2009.
Presented by Alexandru Bleotu
Revisiting read wear: analysis, design, and evaluation of a footprints scrollbar
Jason Alexander, Andy Cockburn, Stephen Fitchett, Carl Gutwin & Saul Greenberg
ACM Human Factors in Computing Systems, April 2009.
Presented by Neil Jenkins
What do you see when you're surfing?: using eye tracking to predict salient regions of web pages
Georg Buscher, Edward Cutrell & Meredith Morris
ACM Human Factors in Computing Systems, April 2009.
Presented by Aloysius Han
Transparent 2D markers on an LCD tabletop system
Hideki Koike, Wataru Nishikawa & Kentaro Fukuchi
ACM Human Factors in Computing Systems, April 2009.
Presented by Alex Toumazis


Participants will be expected to undertake six hours of preparatory work before each meeting. This will involve:

  • Reading the three papers scheduled for consideration that week.
  • Following up references, subsequent citations and other related work.
  • Writing an essay of about a thousand words giving a summary of the three set papers and discussing their broader context.
  • Submitting the essay by noon on the day preceding the meeting.

Every week, three participants will each introduce one of the papers being considered. This will involve giving a 20 minute presentation as if reporting the work at a conference, followed by 5 minutes of questions and 10 minutes of dicussion. The final 15 minutes will be spent discussing the broader issues raised by the week's papers.

Each participant will also prepare an additional report on a paper of their own choice over the Christmas vacation for presentation at a final meeting of the group held at the beginning of the Lent Term. This will probably be on the afternoon of Tuesday 12 January 2010.


  • Written reports on weekly reading [7×8%]
    The reports should be submitted to the Student Administration Office by 14:00 on the day before the relevant meeting of the group.
  • Contributions to weekly discussions [8×3%]
  • Individual presentation at one group meeting [10%]
  • Individual presentation at the Lent Term meeting [10%]

There will be 12 participants and 21 papers to be presented, so most people will present two papers. This obviously involves more work, but the better of the two presentations will count for assessment.


The report should demonstrate that you have read and understood the three papers for the week. This should take the form of an extended abstract for each paper in 200-300 words, explaining the problem being tackled, the approach taken, and reviewing the evaluation. The final section of the report should compare the three papers, exploring their strengths and weaknesses. In the case of older papers, it would be worth explaining their influence on subsequent research.

You should also think of a couple of questions that you would like to ask when the paper is presented.


Present the paper as if it were your own work that you were presenting at a conference, explaining what was significant at the time the paper was published. However, you should also add a coda explaining the influence that the paper had on subsequent research.

Presentations may use any standard audio-visual aids. SS03 has a video projector which can be driven from a workstation in the room, your own lap-top computer, or a visualiser for printed material; there is also an amplifier and loudspeakers. It may be convenient to prepare material with PowerPoint or LaTeX for presentations. These can be delivered using either the workstation in SS03 or from your own lap-top computer.

  • If you want to use the workstation in SS03, please put your presentation on your CL or PWF Web pages, and send me the URL by 14:00 on the Monday before it will be needed. I will then check that it works and confirm later on Monday. Of course, you can continue to change the content until Tuesday afternoon.
    Use a secure file transfer program such as WinSCP to transfer your presentation to the public_html directory in your home directory on (for the CL) or (for the PWF).
  • If you want to user your own lap-top computer, please check that you know how to connect it before the meeting.

Further resources