Shoebox

Introduction

Shoebox is an application for organizing, annotating, indexing, searching, and displaying collections of digital images. It is particularly aimed at collections of personal photographs obtained either from digital cameras or by scanning photographs taken with conventional cameras, but it has also proved useful in a number of other contexts where large volumes of more specialized images need to be organized and indexed.

With the rapidly increasing sophistication, affordability, and popularity of digital cameras, the volume of photographs taken for both personal and vocational use is bound to increase greatly in the near future. Thus, there will be an increased demand for tools to manage large collections of digital images, and this observation has led us to experiment with the ideas embodied in Shoebox. Much of the technology used in Shoebox grew out of the DART project, our wide-ranging research effort in multimedia information retrieval.

Note that Shoebox is still in the research prototype stage and there are no fixed plans to release it as a product yet. Any developments in its availability will be reflected in these web pages, so if you are interested please check back here every now and then.

Features

Shoebox provides three primary features:

a powerful yet simple-to-use thumbnail-based browsing tool for organizing, labelling, and viewing photos.
an audio annotation capability whereby users can speak about their photos. These audio annotations are automatically transcribed to text and an inverted index is derived from this text. Thus, users are able to search for photos based on what they have said about them. The audio annotations are also retained in playable form so that the user can construct canned slide-shows with recorded commentary or simply listen to what they said about their photos.
image analysis and indexing algorithms which allow the user to search for photos based on their visual content, e.g. "find me other images which look like this one", or "find me images which contain regions similar to these selected regions."

OODB

Underlying the user-level features of Shoebox there is an ODMG-compliant object-oriented database (OODB) which was designed and developed here at AT&T Laboratories Cambridge, primarily as a vehicle for experimentation in the field of multimedia information retrieval. Because Shoebox is based on this heavyweight database, it will easily scale to handle tens or even hundreds of thousands of images and annotations.

Wireless Camera

In conjunction with our research and development of Shoebox, we have also prototyped the concept of a wireless camera. To support this concept, Shoebox provides an Inbox feature whereby new photos can be incorporated into a Shoebox database dynamically, much as new email messages are incorporated by email clients. The effect is that as soon as the user snaps a photo, it is immediately and seamlessly transmitted to their Shoebox where it can be labelled, annotated, and indexed.

Ongoing Research

We are continuing to develop and experiment with different schemes for image segmentation and indexing, and Shoebox provides us with an ideal framework for these experiments. Our technique for generating multidimensional indices from the vast amounts of data which our image segmentation algorithms yield is both efficient and scalable and therefore is an improvement upon existing techniques.

Extended mechanisms for database recovery in the OODB are also on our research agenda, as are formal user evaluation studies to try to determine the utility of both the searchable audio annotation and the image searching features of Shoebox.