Modern mobile phones offer much more than just voice communications.
Many popular handsets (e.g. Nokia 3650) are programmable and incorporate
cameras, Internet access, short-range wireless communications via Bluetooth,
and colour displays. The aim of our work is to design and implement novel
interaction methodologies and applications that exploit these devices to
their full potential.
The SpotCode Interfaces project allows people to use their existing
camera-phones to interact with computers in the environment, letting the
phone be used as:
Unlike other universal remote control projects, we accomplish all of this using
existing consumer hardware!
- A sophisticated pointing device that can act as a mouse and understand
- An augmented reality viewer to display information about objects in the
- A user interface for devices without displays or input capability of their
Feeling impatient? Skip straight to the cool videos!
SpotCode Interfaces rely on two core technologies:
The phone software (High Energy Magic's "SpotReader") transmits telemetry about the currently visible tags to the listening computer via Bluetooth. The telemetry includes the 42-bit tag ID, the coordinates of the tag on the phone display, the rotation of the phone relative to the tag, and the perceived size of the tag on the phone (to judge distance).
- Visual tags, accurately detected in real-time by camera-phones using software written by High Energy Magic. These tags can either be active (e.g. generated by a computer and shown on a display), or passive (e.g. printed on a poster).
- Bluetooth is used for short-range wireless communications between a camera-phone and a nearby computer.
The SpotReader detects tags passing across its camera in real-time, and marks the currently selected tag with a red cross (see above picture).
The consumer can now use their phone to interact with active displays or even just printed posters in the environment (e.g. shop-fronts or public kiosks). Why not just use touch-screens for this? Here are a few reasons:
- Cost: The content provider doesn't need to pay to install expensive touch-screens which run the risk of being vandalised or broken. Rather, the displays can be protected, and the consumers use their own technology (their phones) to interact with them. If no active elements are required in the interface, a simple poster can be printed at minimal cost!
- Personalisation: Unlike existing public kiosks, the user can choose to transmit their identity when they use their phone to interact with these services, in a manner very similar to web browser cookies.
- Billing: Mobile phone providers are starting to provide integrated billing services on mobile phones, allowing for easy, trusted purchased to be made. In contrast, there is always a danger of fraud when entering credit card information into public kiosks or at Internet cafes.
- Ease of use: Other pioneering research such as Cooltown has demonstrated that using physical elements (such as SpotCodes) to link users directly to online content can be intuitive and easily understood. Currently, users have to type in cumbersome URLs in order to access online content, which is error-prone and difficult.
- Speed: By using Bluetooth to enable local interaction, users can make decisions using a low-latency, responsive interface (something rather unusual when using WAP sites today!), and only go online to complete the purchase. Another advantage is that the user doesn't incur phone charges for using the local interface, unlike using GPRS or WAP.
- Privacy: Since SpotCodes allow user interface elements to be "thrown" to the users phone (see video below), certain decisions can be made without being exposed on a public screen. Examples of this include entering security PIN numbers or selecting media content.
SpotCodes enable some novel user interface elements. Click on the videos below for a preview!
Throwing the UI
Since the phone computes the relative rotation angle of detected tags in real-time, an intuitive "volume control" is possible by simple locking onto a tag and physically rotating the phone. This video demonstrates a volume control in the "SpotCode Jukebox".
In the "SpotCode Jukebox", the user can skip to a different location in the current track by "grasping" the slider control and moving it to a new location. The music changes in response to the user selection, giving them immediate feedback.
To select a music track to play in the "SpotCode Jukebox", the user just clicks on a tag which then throws a list of tracks to the phone. This video demonstrates a list box, but a variety of elements such as text input and radio boxes are also supported.
A public display at a travel agency displays a world map augmented with SpotCodes, allowing consumers to interact with the display using their cellphone. Click on the pictures below to see the videos!
Note that the user did not have to enter any cumbersome URLs onto their phone at any point - they simply used a local interactive point-and-click interface, which automatically forwarded them onto the Internet to make the purchase via an SMS text message!
Active Plasma Display
A user navigating the world map using a plasma screen. Apologies for the flickering screen on this video - our crazy technical skills didn't extend to finding the "Sync on Green" port on our digital camera.
A good situation where this technology is useful might be a travel agency's shop-front window. Special offers could be on display, and shoppers could purchase them from the outside even when the shop is closed!
The same world-map interface projected onto a wall - now the user can interact with a display with no moving parts in sight to vandalise!
This has a number of great uses - imagine watching a Powerpoint presentation and just "clicking" on a projected slide in order to save a link for future research!
SpotCodes can be read just as easily off a piece of paper as they can from a computer display. To illustrate this, and how incredibly robust they are, check out the videos below.
An example of a passive SpotCode poster - the Spiderman movie soundtrack can be clicked and listened to on with a user's mobile phone, and tracks selected without needing any expensive displays.
Environmental speakers in the shop could preview the sound, and the music could be purchased on-line and transferred to a Bluetooth-enabled music player.
While comparing various tag formats, we noticed that the High Energy Magic SpotCodes are incredibly robust to the paper being smudged or distorted. The above video demonstrates the poster after our "wastebasket test", and as you can see, the tag is still successfully decoded and the interface thrown to the phone. Unfortunately, none of the other tag formats we tried were this robust, probably due to SpotCodes being designed specifically for low-quality mobile phone hardware.
For press enquiries or further information, feel free to get in touch! Demos are also available if you visit Cambridge...
Many thanks to High Energy Magic for licensing the SpotCode Platform to us for research purposes, without which this work would not have been possible.