Figure 1b (Page 2) is used as a motivational experiment to show that Hadoop MapReduce is capable of interfering with the behavour of memcached. These results are repeated in Figure 3b (Page 7) with the addition of QJump to show that QJump is capable of resolving this interference.

Figure 1b

Figure 3b

Software Required

In addition to these general software requirements, we also used:


To obtain more detailed results from memaslap, we patched it to collect statistics in memory and dump them to standard output when complete. The patch file should be applied to the clients sub-directory of the libmemcached 1.0.15 source tree listed above. To build memaslap, run "./configure --enable-memaslap" followed by "make && sudo make install".

Physical Configuration

The basic physical network configuration is described here. In this experiment the following configuration was used:

Sofware Configuration

The full Hadoop Map-Reduce configuration can be found here. In experiments without QJump running, memcached and memaslap were configued as follows:

/usr/bin/memcached -m 64 -p 11211 -u memcache
../clients/memaslap -s -S 1s -B -T2 -c 128 > out

When QJump was enabled, it was configured for memcached and memasalp to run at piority level 3 with a rate limit of 5Gb/s:

bytesq=256 timeq=5 p4rate=100

To run memaslap and memcached at prioirty 3 with QJump enabled, they were run with our application utility:

./ -p 3 -c "../clients/memaslap -s -S 1s -B -T2 -c 128 > out"
./ -p 3 -c "/usr/bin/memcached -m 64 -p 11211 -u memcache"

Raw Data

Our original unprocessed datasets can be found here (93MB). There are 15 files in the archive. Each file contains a short header followed by rows of the form:


The file names are of the form:

Processing and plotting the results

The processing and plotting scripts can be found here. To produce the figure, we first preprocessed the data as follows:

for i in {0..4} ; do ./ ../data/memcachedBS301_hadoop_P3_ $i ; done
for i in {0..4} ; do ./ ../data/memcachedBS301_hadoop_ $i ; done
for i in {0..4} ; do ./ ../data/memcachedBS301_ $i ; done

cat ../data/out_memcachedBS301_[0-4].lats > ../data/out_memcachedBS301_all.lats
cat ../data/out_memcachedBS301_hadoop_[0-4].lats > ../data/out_memcachedBS301_hadoop_all.lats
cat ../data/out_memcachedBS301_hadoop_P3_[0-4].lats > ../data/out_memcachedBS301_hadoop_P3_all.lats

Figure 1b and 3b can then be produced as follows

python \
  ../data/memcachedBS301_all.lats idle \
  ../data/memcachedBS301_hadoop_all.lats contended

python \
  ../data/memcachedBS301_all.lats idle \
  ../data/memcachedBS301_hadoop_all.lats contended \
  ../data/memcachedBS301_hadoop_P3_all.lats QJump