Measuring Burstiness in Data Center Applications
Authors: Jackson Woodruff, Andrew W Moore, Noa Zilberman
Abstract
Buffer sizing is a tricky task, depending on a large number of variables, ranging from congestion control to traffic engineering. Still, the most unpredictable contributors are the workloads running in the network. The link utilization and burstiness of these workload dictate the depth of the buffer we need on a switch. But what is a burst? Do traditional definitions still apply at the age where switches transfer terabits of data and billions of packets every second? Unless we assess bursts correctly, we are unlikely to size buffers appropriately. In this work, we present a measurement-led work evaluating the burstiness of different data center applications. We address the question of ``what is a burst?'' and assert that common techniques can not answer this question. We observe the change in burtiness of the studied applications across multiple vectors, including latency and number of machines, and generalize our results to the common case. Our observations can inform future buffer sizing works and guide switch configurations. We are making our dataset openly available for the benefit of the community.