|
| The Origami Project |
Published in the proceedings of 7th International Conference on Human-Computer Interaction, San Fransisco, August 1997. ©Elsevier Science BV 1997. Reproduced with permission.
[1] University of Cambridge, Computer Laboratory, New Museums Site, Pembroke Street, Cambridge, England CB2 3QG.
[2] University of Cambridge, Department of Applied Mathematics and Theoretical Physics, Silver Street, Cambridge, England CB3 9EW.
This paper reports on ways of using digitised video from television cameras in user interfaces for computer systems [Robinson 1995]. The DigitalDesk [Wellner 1993, Wellner 1994] shown in Figure 1 is built around an ordinary physical desk and can be used as such, but it has extra capabilities. A video camera mounted above the desk, pointing down at the work surface, is used to detect where the user is pointing and to read documents that are placed on the desk. A computer-driven projector is also mounted above the desk, allowing the system to project electronic objects onto the work surface and onto real paper documents.
In particular, this technology has been applied to electronic publishing. The goal is to combine electronic and printed documents to give a richer presentation than that afforded by either separate medium. We introduce the idea of combined documents, show how they can be used for mixed-media publication and discuss some of the technical issues involved.
Electronic, multi-media publishing is becoming established as an alternative to conventional publishing on paper. CD-ROM and on-line versions of reference books and fiction can augment their conventional counterparts in a number of ways:
However, screen-based documents have a number of disadvantages:
Our solution is to publish material as an ordinary, printed document that can be read in the normal way, enjoying the usual benefits of readability, accessibility and portability. However, when observed by a camera connected to a computer, the material acquires the properties of an electronic document, blurring the distinction between the two modes of operation.
Our initial experiments have applied this technology to computer-assisted learning. Earlier work with Computer Illustrated Texts [Harding & Quinney 1990] supplemented printed books with software that was an integral part of the educational package but which had to be run separately. The two parts can now be united.
Consider an elementary textbook on mathematics that is discussing curve sketching for polynomials. The printed page shows a generic equation with blank boxes for the coefficients. When values are written into the boxes, the corresponding graph is projected into another blank box on the page. The same system can also be used for assessment. The computer projects coefficients and the student sketches the curve which is then analysed by the computer. Figure 2 shows a page of the text book with the interaction inset [New technology for interactive CAL: The Origami Project].
Figure 1: The DigitalDesk in use.
Figure 2: Animated maths book.
More recently, the technology has been used to animate a standard printed book using additional information from the British National Corpus of digital editions of modern English texts to provide the electronic annotations [Active paper for active learning, Active Alice - using real paper to interact with electronic text]. The system has also been used to provide an interface to the World-Wide Web via printed paper [Paper interfaces to the World-Wide Web] as shown in Figure 3 and to provide a paper interface to electronic mail.
Three areas in the infrastructure have been investigated [A Framework for Interacting with Paper]. The first is video resolution. New projectors and cameras are becoming available that allow laser printer resolution for desktop image projection and capture. We have been experimenting with a variable resolution camera that uses a prismatic lens to sample at up to six times the resolution of its CCD array.
Secondly, there is the question of character and image: the printed document is the anchor for all the enhancements, but it is necessary to deliver these differently depending on whether the original is available in electronic form or only appears on paper.
Finally, the user interface is being extended to exploit the new facilities. The idea is to move away from explicit commands from a menu and towards a scheme in which the camera observes the activities of the user and infers the appropriate assistance to offer. Simply picking up a pencil or eraser or reference book should cause the program to react.
We have investigated two document structures, one based on SGML and one based on active objects. Both provide links to other documents and to the programs implementing annotations. These can be held in a separate registry which allows links embedded in a paper document to be activated remotely.
In use, the system resembles a Web browser working from paper rather than a screen. Links can be activated by pointing with a stylus and forms can be completed simply by writing.
An editor for mixed-media documents has been written which allows text and images to be copied from paper documents. This is loosely follows the model of Reno [Kerr et al 1995], with a small set of fundamental objects and behaviours imposed by grouping. Links can also be constructed simply by selecting the active area of the new document. This can then be printed in a form suitable for recognition by the system.
| 
| 
|
| (a) Original Web page. | (b) Printed version. | (c) Animated on the DigitalDesk. |
| 
| 
|
| (d) Deriving a new document. | (e) Printed version. | (f) Animating the new document. |
Figure 3: Animating a paper document on the DigitalDesk.
Figure 3 shows the system in use. A conventional WWW page is shown at (a). This is imported into the system and reprinted with additional coding to assist recognition (b). When this is placed on the DigitalDesk it is recognised and active areas of the document illuminated by projected highlights. When these are selected, links are followed or programs executed and the results projected into a further window on the work surface (c).
Moreover, fragments can be copied from the paper document into new electronic documents also projected onto the desk (d). The new document can be printed to give a new paper document (e) which can be animated on the desk in just the same way (f).
We have developed the underlying technology of the DigitalDesk and used it as a vehicle to demonstrate mixed-media documents adding electronic annotations to printed pages. This has prepared the way for widespread use of animated paper documents as the relevant hardware becomes more generally available.
Further work on video user interfaces has investigated the use of cameras alone, without projection [Stafford-Fraser 1996, Stafford-Fraser & Robinson 1996]. We are also looking at the use of this to animate printed documents.
Home | Background | Publications | References | Places of Interest | Resources