Computer Laboratory

Course material 2010–11

Innovative user interfaces

Reading group for the MPhil in Advanced Computer Science.
Peter Robinson.
Michaelmas Term, Tuesday, 16:00-18:00, SS03.


This module aims to provide a review of innovative user interfaces.

On completing the module, students should understand the development of user interfaces throughout the history of computing, and be aware of recent research directions.


The module consists of eight two-hour meetings of a reading group.

Each participant will be expected to read some relevant papers about each topic, and write an essay on it to be submitted before the meeting. Each participant will also be expected to give presentations about two of the weekly topics to the other members of the class.


It is assumed that students will have taken an undergraduate course in human-computer interaction and will be familiar with the standard literature such as:

Interactive system design
William Newman & Mik Lamming.
Addison-Wesley, 1995.

Preliminary reading

Students intending to take the course should read the following before the beginning of term:

The psychology of everyday things
Donald Norman.
Basic Books, 1988.
The computer for the 21st century
Mark Weiser.
Scientific American, September 1991.
Reprinted in Mobile Computing and Communications Review, July 1999.
The intuitive beauty of computer-human interaction
Communications of the ACM 43(3), March 2000.


Week 1: Early inspiration

Participants do not need to submit an essay in advance of the first meeting, but should still read the following papers and come prepared to discuss them.

As we may think
Vannevar Bush.
Atlantic Monthly, July 1945.
Man-computer symbiosis
Joseph Licklider.
IRE Transactions on Human Factors in Electronics, March 1960.
Ivan Sutherland.
AFIPS Spring Joint Computer Conference, 1963.
Full dissertation reprinted as CL Technical report, September 2003.
Videos 1/2 and 2/2.
A methodology for user interface design
Charles Irby, Linda Bergsteinsson, Thomas Moran, William Newman & Larry Tesler.
Xerox PARC, 1977.

Week 2: Windows, icons, mice and pointing

The model human processor
Stuart Card, Tom Moran & Alan Newell.
Handbook of Perception and Human Performance, Wiley, 1986.
Star graphics: an object-oriented implementation
Daniel Lipkie, Steven Evans, John Newlin & Robert Weissman.
ACM SIGGraph 16(3), July 1982.
Videos 1/2 and 2/2.
The X Window System
Robert Scheifler & Jim Gettys.
ACM Transactions on Graphics 5(2), April 1986.

Week 3: Video user interfaces

Interacting with paper on the DigitalDesk
Pierre Wellner.
Communication of the ACM 36(7), July 1993.
BrightBoard: A video-augmented environment
Quentin Stafford-Fraser & Peter Robinson.
ACM Human Factors in Computing Systems, April 1996.
High-resolution interactive displays
Mark Ashdown, Philip Tuddenham & Peter Robinson.
Tabletops — Horizontal Interactive Displays, Springer, 2010.

Week 4: Direct manipulation

Direct manipulation: a step beyond programming languages
Ben Shneiderman.
IEEE Computer 16(8), August 1983.
Toolglass and magic lenses: the see-through interface
Eric Bier, Maureen Stone, Ken Pier, William Buxton & Tony DeRose.
ACM SIGGraph, August 1993
Tangible bits: beyond pixels
Hiroshii Ishii.
ACM Conference on Tangible and Embedded Interaction, February 2008.

Week 5: The disappearing computer

Some Computer Science issues in ubiquitous computing
Mark Weiser.
Communications of the ACM 36(7), July 1993.
Sentient computing
Andy Hopper.
Philosophical Transactions of the Royal Society A 358, August 2000.
Building disappearing computers
Daniel Russel, Norbert Streitz & Terry Winograd.
Communications of the ACM 48(3), March 2005.

Week 6: Special purposes

What's real about virtual reality?
Frederick Brooks.
IEEE Computer Graphics and Applications 19(6), November 1999
Bridging the physical and digital in pervasive computing
Steve Benford, Carsten Magerkurth & Peter Ljungstrand.
Communications of the ACM 48(3), March 2005.
Pragmatic research issues confronting HCI practitioners when designing for universal access
Simeon Keates.
Universal Access in the Information Society 5(3), November 2006.

Week 7: Affective computing

Facial Expression and Emotion
Paul Ekman.
American Psychologist 48(3), April 1993.
Affective Computing
Rosalind Picard.
MIT Media Lab, 1995.
Computers that care
Scott Brave, Clifford Nass & Kevin Hutchinson.
International Journal of Human-Computer Studies 62, January 2005.

Week 8: Emotional inference

Real-time inference of complex mental states from facial expressions and head gestures
Rana el Kaliouby & Peter Robinson.
Real-time vision for HCI, Springer-Verlag, 2005.
Classification of complex information: Inference of co-occurring affective states from their expressions in speech
Tal Sobol-Shikler & Peter Robinson.
IEEE Transactions on Pattern Analysis and Machine Intelligence 31, May 2009.
Detecting affect from non-stylised body motions
Daniel Bernhardt & Peter Robinson.
International Conference on Affective Computing and Intelligent Interaction, September 2007.


Participants will be expected to undertake six hours of preparatory work before each meeting. This will involve:

  • Reading the three papers scheduled for consideration that week.
  • Following up references, subsequent citations and other related work.
  • Writing an essay of about a thousand words giving a summary of the three set papers and discussing their broader context.
  • Submitting the essay by noon on the day preceding the meeting.

Every week, three participants will each introduce one of the papers being considered. This will involve giving a 20 minute presentation as if reporting the work at a conference, followed by 5 minutes of questions and 10 minutes of discussion. The final 15 minutes will be spent discussing the broader issues raised by the week's papers.

Everyone should also think of a couple of questions that could be asked when the paper is presented.


The report should demonstrate that you have read and understood the papers for the week. This should:

  • Explain the context of the three papers.
  • Summarise their contributions in 200-300 words each, explaining the problem being tackled, the approach taken, and reviewing the evaluation.
  • Compare the three papers, exploring their strengths and weaknesses. In the case of older papers, it would be worth explaining their influence by identifying their impact on subsequent work and identifying any omissions that inspired subsequent work.
  • Cite references to support the analysis.


Present the paper as if it were your own work that you were presenting at a conference, explaining what was significant at the time the paper was published. However, you should also add a coda explaining the influence that the paper had on subsequent research.

Presentations may use any standard audio-visual aids. SS03 has a video projector which can be driven from a workstation in the room, your own lap-top computer, or a visualiser for printed material; there is also an amplifier and loudspeakers. It may be convenient to prepare material with PowerPoint, LaTeX, Prezi or some such for presentations. These can be delivered using either the workstation in SS03 or from your own lap-top computer.

  • If you want to use the workstation in SS03, please put your presentation on your CL or PWF Web pages, and send me the URL by 14:00 on the Monday before it will be needed. I will then check that it works and confirm later on Monday. Of course, you can continue to change the content until Tuesday afternoon.
    Use a secure file transfer program such as WinSCP to transfer your presentation to the public_html directory in your home directory on (for the CL) or (for the PWF).
  • If you want to user your own lap-top computer, please check that you know how to connect it before the meeting.


  • Written reports on weekly reading [7×8%]
    The reports should be submitted to the Student Administration Office by noon on the day before the relevant meeting of the group.
  • Contributions to weekly discussions [8×3%]
  • Individual presentations at two group meetings [2×10%]

There will be 10 participants and 21 papers to be presented, so most people will present two papers. If someone gives more than two presentations, the best two marks will count.

Further resources