Figure 1b is used to show that Hadoop MapReduce is capable of interfering with the behavour of memcached and that QJump is capable of resolving this interference.
In addition to these general software requirements, we also used:
To obtain more detailed results from memaslap, we patched it to collect statistics in memory and dump them to standard output when complete. The patch file should be applied to the clients sub-directory of the libmemcached 1.0.15 source tree listed above. To build memaslap, run "./configure --enable-memaslap" followed by "make && sudo make install".
The basic physical network configuration is described here. In this experiment the following configuration was used:
The full Hadoop Map-Reduce configuration can be found here. In experiments without QJump running, memcached and memaslap were configued as follows:
/usr/bin/memcached -m 64 -p 11211 -u memcache
../clients/memaslap -s 10.10.0.6:11211 -S 1s -B -T2 -c 128 > out
When QJump was enabled, it was configured for memcached and memasalp to run at piority level 3 with a rate limit of 5Gb/s:
bytesq=256 timeq=5 p4rate=100
To run memaslap and memcached at prioirty 3 with QJump enabled, they were run with our application utility:
./qjau.py -p 3 -c "../clients/memaslap -s 10.10.0.6:11211 -S 1s -B -T2 -c 128 > out"
./qjau.py -p 3 -c "/usr/bin/memcached -m 64 -p 11211 -u memcache"
Our original unprocessed datasets can be found here (93MB). There are 15 files in the archive. Each file contains a short header followed by rows of the form:
{GET | SET | TOTAL} , REQUEST ID, LATENCY
The file names are of the form:
The processing and plotting scripts can be found here. To produce the figure, we ran the following bash script:
for i in {0..4} ; do ./do_hist.sh memcachedBS301_hadoop_P3_ $i ; done
for i in {0..4} ; do ./do_hist.sh memcachedBS301_hadoop_ $i ; done
for i in {0..4} ; do ./do_hist.sh memcachedBS301_ $i ; done
cat out_memcachedBS301_[0-4].lats > out_memcachedBS301_all.lats
cat out_memcachedBS301_hadoop_[0-4].lats > out_memcachedBS301_hadoop_all.lats
cat out_memcachedBS301_hadoop_P3_[0-4].lats > out_memcachedBS301_hadoop_P3_all.lats
python plot_memcached_latency_cdfs.py memcachedBS301_all.lats idle memcachedBS301_hadoop_all.lats contended memcachedBS301_hadoop_P3_all.lats QJump