Technical reports

A wide-area file system for migrating virtual machines

Tim Moreton

March 2008, 163 pages

This technical report is based on a dissertation submitted February 2007 by the author for the degree of Doctor of Philosophy to the University of Cambridge, King’s College.

DOI: 10.48456/tr-714

Abstract

Improvements in processing power and core bandwidth set against fundamental constraints on wide-area latency increasingly emphasise the position in the network at which services are deployed. The XenoServer project is building a platform for distributed computing that facilitates the migration of services between hosts to minimise client latency and balance load in response to changing patterns of demand. Applications run inside whole-system virtual machines, allowing the secure multiplexing of host resources.

Since services are specified in terms of a complete root file system and kernel image, a key component of this architecture is a substrate that provides an abstraction akin to local disks for these virtual machines, whether they are running, migrating or suspended. However, the same combination of wide-area latency, constrained bandwidth and global scale that motivates the XenoServer platform itself impedes the location, management and rapid transfer of storage between deployment sites. This dissertation describes Xest, a novel wide-area file system that aims to address these challenges.

I examine Xest’s design, centred on the abstraction of virtual disks, volumes that allow only a single writer yet are transparently available despite migration. Virtual disks support the creation of snapshots and may be rapidly forked into copies that can be modified independently. This encourages an architectural separation into node-local file system and global content distribution framework and reduces the dependence of local operations on wide-area interactions.

I then describe how Xest addresses the dual problem of latency and scale by managing, caching, advertising and retrieving storage on the basis of groups, sets of files that correspond to portions of inferred working sets of client applications. Coarsening the granularity of these interfaces further decouples local and global activity: fewer units can lead to fewer interactions and the maintenance of less addressing state. The precision of these interfaces is retained by clustering according to observed access patterns and, in response to evidence of poor clusterings, selectively degrading groups into their constituent elements.

I evaluate a real deployment of Xest over a wide-area testbed. Doing so entails developing new tools for capturing and replaying traces to simulate virtual machine workloads. My results demonstrate the practicality and high performance of my design and illustrate the trade-offs involved in modifying the granularity of established storage interfaces.

Full text

PDF (1.7 MB)

BibTeX record

@TechReport{UCAM-CL-TR-714,
  author =	 {Moreton, Tim},
  title = 	 {{A wide-area file system for migrating virtual machines}},
  year = 	 2008,
  month = 	 mar,
  url = 	 {https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-714.pdf},
  institution =  {University of Cambridge, Computer Laboratory},
  doi = 	 {10.48456/tr-714},
  number = 	 {UCAM-CL-TR-714}
}