The Pegasus project developed both a file server and a file-server simulator. The Pegasus file server is based on a client-server architecture where caching, buffering, and (RAID) storage are all designed to prevent data loss in any single-point hardware failure.
In Pegasus II, the work on the file server is being extended to incorporate dependable archival storage and predictable multimedia storage and retrieval. The work makes available on Nemesis a file system interface which is Posix compliant, and will use Nemesis as the platform for delivering the archival storage service. The archival storage service will allow a single file system to manage up to 10 terabytes of storage on storage juke boxes and retrieve information with a minimum of time-consuming disk or tape changes. The current single-fault tolerance of the file system will be extended to the archival part of the file system, so that loss of any one (optical) disk, tape, or CD-ROM will not result in loss of data. The multimedia-storage extensions will provide service guarantees for up to ten concurrent video and audio streams. In addition this workpackage will deliver Posix compatible interfaces to the project.
This work package is being done at Twente. It comprises the following tasks:
(Twente)
When video and audio are being recorded, the system must provide guarantees that there is sufficient bandwidth from camera and microphone to the storage media so that no data is lost. Similarly, when audio and video data are being played back from the storage system, there must be sufficient bandwidth to prevent gaps in the output stream from occurring.
When a video or audio stream, to or from the storage server, is created, the system must reserve resources to guarantee that the stream can be given the service it requires. The accommodation of multiple, simultaneous streams requires careful scheduling and buffer allocation.
The goal of this task is to design the buffer management and scheduling policies that allow the Pegasus storage server to guarantee service for up to ten simultaneous continuous-media streams. The remaining server bandwidth must be sufficient for a reasonable best-effort performance for non-real-time (conventional) file-server traffic. It is our intention to develop a buffer management and scheduling scheme that optimizes the use of the storage I/O system.
We are also investigating multimedia file storage and playback from our archive. The main difference with the Hermes project is that our environment is more flexible as more users can change the archive. This influences the mulitmedia data layout policies on the archive. Where possible are using technology developed by the Hermes project for our system (e.g., client admission control and bandwidth reservation schemes).
Deliverables:
(Twente)
The Pegasus File Server will run on Unix and Nemesis platforms and also be capable of serving clients on either Unix or Nemesis. On both Unix and Nemesis the interfaces to the file server will be identical, and Posix-compatible. Behind each interface client software will maintain a consistent cache (the server will invalidate the client cache in the event of updates by another client), and will double-buffer data to ensure that a failure on the part of either the client or the server will not result in data loss. Both stubs will provide a Posix-compatible interface.
The Pegasus File Server will also export an NFS interface so that unmodified systems (e.g. Unix and PCs, using a simple extension for PC-NFS) can access the Pegasus File Server. The NFS interface, however, does not provide the semantics necessary for consistent caching, nor does it provide the context for single-fault tolerance.
We will make sure that the Pegasus II File Server can easily be integrated into most system configurations for ordinary storage.
We will also make available a library through which the multimedia capabilities of the Pegasus II File Server can easily be used. We will make these interfaces available to other storage projects (e.g., the Hermes ESPRIT project).
Deliverables:
(Twente)
The Pegasus II File Server archive will be implemented on high-density media that is accessed by drives making use of robots to load and unload the data carriers. As preparation for the project, an evaluation is being carried out in collaboration with Philips Research in Eindhoven to find what media provide the best trade-offs in terms of cost, reliability, physical volume, and access time per unit of storage. Results from this study will determine the particular type of tertiary storage media to be used in the Pegasus II archive.
A single archive must scale to a size of tens of terabytes. This implies that the data structures for managing and finding files in the archive must be designed so that a very large increase in systems size will only cause a moderate increase in lookup and access times. In addition, the data structures must provide automatic mechanisms that allow data on deteriorating data carriers to be copied to fresh ones. As an extra guard against data loss, the storage server will use technology similar to that in RAID to prevent loss of a single carrier from losing information.
The Hermes ESPRIT project is developing a digital library system. A digital library is best described as an "electronic library". This includes the addition of catalogue information to allow retrieval of data by a large group of (possibly anonymous) people. Additions to this library are usually done by a librarian, who also produces catalogue information for the digital library. Such systems are mainly used by internet service providers for electronic business.
For Pegasus II we are considering an archive that will be used by a smaller community (e.g. a University faculty) for backup facilities and multimedia file storage.
The main difference between Hermes and Pegasus II is that our system must be more dynamic in terms of updates. Since Hermes is a digital library, there are only a few administrators who update the library and a large user community that accesses the library for data retrieval. In our case there are many users that update and search the archive. This means that algorithms designed by project Hermes may not be efficient for Pegasus II. We intend to design and implement efficient storage and retrieval algorithms for our environment. Where possible, however, we are re-using technology developed by the Hermes project.
Deliverables: