Implementing Declarative Overlays

Overlay networks are used today in a variety of distributed systems ranging from file-sharing and storage systems to communication infrastructures. Overlays of various kinds have recently received considerable attention in the networked systems research community, partly due to the availability of the PlanetLab planetary-scale application platform. However, a broader historical perspective is that overlay functionality has implicitly long been a significant component of wide-area distributed systems.

Despite this, designing, building and adapting these overlays to an intended application and the target environment is a difficult and time consuming process.

To ease the development and the deployment of such overlay networks, my research group at Intel Berkeley in conjunction with the University of California at Berkeley is building P2, a system which uses a declarative logic language to express the overlay networks in a highly compact and reusable form. P2 can express a Narada-style mesh network in 13 rules, and the Chord structured overlay in only 35 rules. P2 directly parses and executes such specifications using a dataflow architecture to construct and maintain the overlay networks. I'll describe the P2 approach, how our implementation works, and give some experimental results showing that the performance and robustness of P2 overlays is acceptable.