The DAN Architecture

A Desk Area Network based machine is similar to current machines in having a number of processors, memory modules, and a variety of peripherals. However, the DAN applies a system wide multiplexing mechanism and channel identification to the interconnection of these modules based on ATM cells; hence the physical units transferred are cells, and they are associated with channels and routed by the labels in their headers.

A multimedia workstation based on a DAN is shown later in the paper in figure 4. Every device is equipped with an interface to the ATM interconnect. Such an attachment is no more complicated than the equivalent bus interface (and even simpler than over-complicated buses such as EISA). One reason for building the prototype DAN, rather than simply pontificating about it, is to illustrate this point. Furthermore, the internal ATM connection does not need to include a complete transmission system, with bit serial communication, clock recovery, clock alignment, fault isolation, buffering etc. Synchronous clocked unidirectional parallel buses can be used due to the physical proximity of the devices. As with standard peripherals, the interface logic may be designed with a specific task in mind and can therefore be as small as a few PALs and a pair of FIFO memories.

ATM as an interconnect

Using ATM as the multiplexing technique within the workstation has similar advantages to the use of ATM in larger networks. Different channels (even multiple channels between the same two devices) can easily have associated with them different characteristics, the channel identifier providing an index which may be used to select contention priority and queuing disciplines, and thus different Qualities of Service.

As already mentioned, many bus based systems (e.g. TURBOchannel, PCI and SBUS), which support an arbitrary sized data transfer between devices, insist that the bus be made available at regular intervals for preemption. Often the particular interval chosen is based on that of the cache / memory transaction. However, using any small fixed-size data transfer unit also achieves this goal. The choice we have made for the DAN is aimed at streamlining the passage of data through the network interface; hence we choose the same size as has been standardised for the local and wide area ATM networks. As will be seen, the choice of 48 data bytes fits well with the implementation of 32 byte cache line transfers when due consideration is given to the bits for the transaction type, and the related read and write addresses.

The use of fixed size transfer units introduces the need to segment larger blocks of data at the source and re-assemble them at the destination. In the case of a single communication this is trivial, but the scheduling of the interconnect can cause multiple blocks to be arriving in an interleaved manner. The reassembly operation may therefore be fairly costly. Many ATM network interface designs provide hardware assistance for conversion to standard protocols. In the DAN it is often sensible to implement device (or media) specific ATM adaptation layers within the device rather than requiring the host interface or the main CPU to implement a plethora of different adaptation layers.

In the wide area statistical multiplexing of a large number of streams is claimed to allow the network to function at high utilisation and support quality of service guarantees, albeit statistical ones; these ``guarantees'' are based on assumptions of independence of the sources and are usually only asymptotically valid. Such arguments clearly do not apply in the desk area where the number of streams is small and where they are often highly correlated. Within the DAN, the management entity for the switch is the operating system, and the required scheduling of the interconnect is tightly coupled to the scheduling of processes on the processor(s). The integration of these scheduling mechanisms, and the treatment of the interconnect as generic operating system resource is under study. Several other groups are also considering scheduling an interconnect where statistical predictions cannot be made with great confidence. The AN2 local area ATM switch uses a combination of static allocation and dynamic computation to form a schedule for every cell time [\bf\protect\citenameAnderson93]. The SYMPHONY architecture for a multimedia workstation [\bf\protect\citenameBovopoulos93], which uses separate main and multimedia buses, also suggests controlled scheduling of the multimedia interconnect.

Implementation choices

The DAN is intended to be the internal interconnect of a machine, so, in common with bus based systems, the simplifying assumption is made that the hardware provides a reliable path for communication between devices. This may of course involve all the techniques seen in bus based systems to ensure a sufficient level of reliability; for example, parity, ECC, transaction retry (hence blocking), etc. The benefit of this reliability is to greatly simplify the control of the system - no need for a 7 layer stack of protocols to read a register on a device. This trade off is made for the DAN as for bus based systems; as the technology and complexity are similar, so similar reliability can be expected.

The exact implementation of the ATM interconnect is not a concern of the DAN architecture. Just as rings [\bf\protect\citenameTennenhouse89], crossbars [\bf\protect\citenameAnderson93][\bf\protect\citenameLeslie91] and buses [\bf\protect\citenameFraser92] have all been used in ATM switches, so they may be used in a DAN machine. It is the transfer of fixed-sized cells and the asynchronous multiplexing that are the key features. However, the reliability requirement, which is effectively that the source must know when to retry a cell, rules out a few ``hail-mary'' switch architectures, which silently drop cells internally.

As with network switches, the use of a crossbar or other space division fabric will enable concurrent data transfer between different pairs of devices, but it may well prove that a high speed slotted bus would provide a cost effective solution for a small number of nodes. As the architecture allows this variation in implementation, certain functions (e.g. cache coherency) are not defined as part of the DAN architecture, rather they are implementation dependent.

Device control

The label size defined for the standard ATM cell is sufficiently small that it is normal to use ATM as a connection oriented technique; hence, before use, context indexed by the label is established in each component of the network traversed by the channel. This is also true within the DAN, where a context is established within devices and, if the stream is passing between a device and the network, within the network interface and network.

Thus devices receiving data can use the VCI to access any channel specific information; the example of a VCI per window, where the context is the coordinates of the window and clip mask, is described in the framestore section. Use of this technique will be seen many times in the description of the implemented DAN based machine.

Within the DAN, the OS is responsible for access control and protection and, if it is doing its job properly, can ensure that only trusted components configure nodes for communications; hence internally, DAN nodes can be relied upon not to use bogus VCIs or violate their bandwidth allocation. The parallels can be seen here with the management of many traditional operating system resources; for example, the discs implementing the backing store of a virtual memory system are controlled by privileged code and then trusted not to DMA to random memory locations.

The use of virtual circuits requires that some entity perform connection management for the DAN nodes. There are three classes of devices:

• ``Dumb'' devices are only able to manage data and an extremely simple control interface, such as the ability to load internal registers from single cell messages on a predefined VCI. The source of such control messages would be a trusted manager on some other node of the DAN. The controller has responsibility for allocating VCIs for data streams and configuring the device. Usually the manager would be outside the data path so that it is likely that the control functions can easily be performed by one of the processing engines without undue interference with its other jobs.

• ``Supervised'' devices have some local processing power which will enable them to perform some local management, for example they might allocate their own VCIs. Again the main connection establishment functions are performed by a more competent processor node, but the control interactions between manager and device are at a higher level.

• ``Smart'' devices have substantial processing power and are able to perform all of their own management functions; some of these nodes will also act as proxies for interactions between ``dumb'' devices and the network and operating system.

Device control and reliability are the main areas for contrast between the DAN and projects such as the ORL Medusa [\bf\protect\citenameGlauert93], where devices are connected to an ATM LAN. On the LAN a greater amount of processing power must be put into devices to manage control and security functions. An office with an ATM LAN switch connecting multimedia devices and a workstation appears very like a DAN based machine but the distinction from an engineering viewpoint is important.