Figure 1a (page 2) is used as a motivational experiment to show that Hadoop MapReduce is capable of interfering with the behavour of precision time protocol. This figure is repeated in Figure 5 (page 8) in a slightly different form, combined with results from memcached combined. In this case, the figure shows that QJump is capable of resolving interference in PTPd as well as memchaced.
In addition to these general software requirements, we also used:
The basic physical network configuration is described here. In this experiment the following configuration was used:
To obtain more detailed results from memaslap, we patched it to collect statistics in memory and dump them to standard output when complete. The patch file should be applied to the clients sub-directory of the libmemcached 1.0.15 source tree listed above. To build memaslap, run "./configure --enable-memaslap" followed by "make && sudo make install".
The full Hadoop Map-Reduce configuration can be found here. The PTPd client and server were configured as follows:
e=1; p=0; ./qjau.py -p $${p} -c "ptpd -c -b eth6.2 -y 0 -D -h -T 10"
e=0; p=7; ./qjau.py -p $${p} -c "ptpd -x -c -g -D -b eth0 -h -T 10 -f ../data/ptpdCFRS201_memcachedBS206_hadoop_R2D2P$${p}_PTPD_$${e}
Where "$$e" is th experiment number and "$$p" is the QJump prioirty level.
Memcached and memaslap were configued as follows:
e=1; p=0; sudo ./qjau.py -p $${p} -c "/usr/bin/memcached -m 64 -p 11211 -u memcache"
e=1; p=0; for {1..25}; do
./qjau.py -p $${p} -c "../clients/memaslap -s 10.10.0.3:11211 -S 1s -B -T2 -c 128 -X 1024" > ../data/ptpdCFRS201_memcachedBS206_hadoop_R2D2P$${p}_$$(printf "%02i" $${i})_MEMD_$${e};
done
Where "$$e" is th experiment number and "$$p" is the QJump prioirty level.
When QJump was enabled, it was configured as follows
bytesq=256 timeq=5 p0rate=1 p4rate=100 p7rate=9999999
Our original unprocessed datasets can be found here (109kB). Inside the archive are file names of form:
2014-01-22 19:51:27:363605, slv, 90e2bafffe27fbc8/01, 0.000140635, -0.000019134, 0.000197000, 0.000108000, 21106
Where the first column is the timestamp, and the fifth coloum is the syncornization offset.
Our original unprocessed datasets can be found here. There are 10 archives of approx 310MB each. The file names are of the form:
{GET | SET | TOTAL} , REQUEST ID, LATENCY
The processing and plotting scripts can be found here. To produce Figure 1, we ran the following:
python plot_ptp_offset_timeline.py \
../data/ptpd/ptpdCFRS201_R2D2P0_PTPD_0 \
../data/ptpd/ptpdCFRS201_memcachedBS206_hadoop_R2D2P0_PTPD_3 \
figure1a.pdf
Before plotting Figure 5, the memcached data needs to be processed into a more compact form. To do so, we ran the following:
python process_mem.py ../data/memcachedBS206_R2D2P0_MEMD_1/
Then, to produce Figure 5, run something like the following:
python plot_ptp_memcached_hadoop_timeline.py \
../data/ptpd/ptpdCFRS201_R2D2P0_PTPD_0 \
../data/memd/ptpdCFRS201_R2D2P0_PTPD_0/memcachedBS206_R2D2P0_MEMD_1.set.processed \
../data/ptpd/ptpdCFRS201_memcachedBS206_hadoop_R2D2P0_PTPD_3 \
../data/memd/ptpd_memcached_haroop_data/ptpdCFRS201_memcachedBS206_hadoop_R2D2P0_MEMD_1.set.processed \
../data/ptpd/ptpdCFRS201_memcachedBS206_hadoop_R2D2P7_PTPD_1 \
../data/memd/ptpd_memcached_haroop_data/ptpdCFRS201_memcachedBS206_hadoop_R2D2P5_MEMD_4.set.processed \
figure5.pdf