Figure 10 (page 10) show's the behvaiour of memcached, both in terms of latency and throughput, as it is subjected to varying degrees of rate limiting by the QJump rate limiter.

Figure 10

Software Required

In addition to these general software requirements, we also used:

• QJump kernel module
• QJump application utility
• Memcached 1.4.14 (installed from Ubuntu packages)
• memaslap from libmemcached 1.0.15 (patched as below)

Patches

To obtain more detailed results from memaslap, we patched it to collect statistics in memory and dump them to standard output when complete. The patch file should be applied to the clients sub-directory of the libmemcached 1.0.15 source tree listed above. To build memaslap, run "./configure --enable-memaslap" followed by "make && sudo make install".

Physical Configuration

The basic physical network configuration is described here. In this experiment the following configuration was used:

• H3 ran the (patched) memaslap client
• H11 ran the memcached server

Sofware Configuration

This experiment requires iterating over a collection of parameters to the QJump kernel module, memcached and others. We have automated this to make it easy to run. The automation scripts can be found here. There are three scripts that automate the experiment.

• run_memcached - This is the coordinator script and ideally runs on a coordination host, not part of the cluster. It runs the client and server scripts on the client and server hosts.
• run_memcached_cleint - This runs on the client. It configures QJump and runs the memcached client (memaslap) and stores the results.
• run_memcached_server - This runs on the server. It configures QJump and runs the memcached server and stores the results.

Raw Data

Our original unprocessed datasets can be found here (~170MB ea). There are 7 archives. Each archive is for a different configuration. We have run this experiment in several configurations.

• with window size / no window size - with or without TCP tx/rx buffer size restrictions to minimize TCP retransmit timeouts
• with TSO / no TSO - With or without the TCP Segement offload feature enabled on the NIC
• with RTO 5ms - Linux kernel reconfigured for a 5ms RTO instead of a 200ms RTO as default

Each archive contains files of the following form:

• memd_prio_client_[1 ... 1000] - memaslap output
• memd_prio_client_[1 ... 1000].README - a complete list of the commands issued to run the experiment
• memd_prio_client_[1 ... 1000].syslog - the system log output showing the configuration of the kernel during

The majoirty of the data is found in the memasalp output files. These files include the output of the qjau.py application as well as the raw data. Raw data has the following form:

{GET | SET | TOTAL} , REQUEST ID, LATENCY

Processing and plotting the results

For the figure presented in the paper we used the "default" configuration. Which is QJump enabled, no windowing limitations and TSO enabled. The processing and plotting scripts can be found here. To produce the figure, we first preprocessed the data as follows:

files= $(for i in 10 25 50 75 90 100 250 500 750 900 1000 2500 5000; echo -n "../data/memd_rate_data/no_window_size_with_tso/memd_prio_client_$i $i; done; )
python plot_memcached_throughput_latency_comparison.py "$files"