After downloading the data from the website, I mangled it with a Python script into a format more amenable to analysis: The Observation column represents the search query, it's a string value so that R interprets it as a categorical variable. Day,Machine,Observation,Time Monday,alpha,obs1,619 Monday,alpha,obs2,440 Monday,alpha,obs3,555 Monday,alpha,obs4,647 Monday,alpha,obs5,582 Monday,alpha,obs6,317 Monday,alpha,obs7,442 Monday,nu,obs1,677 ... This was loaded into R, the transcript of that session is below (with mistakes, redundancy, etc. edited out). The data file used is available at: http://www.cl.cam.ac.uk/~sd601/data.csv This file is available at: http://www.cl.cam.ac.uk/~sd601/log.txt sd601@chelmer:/auto/homes/sd601\$ R R version 2.14.1 (2011-12-22) > data <- read.csv("~/Documents/data.csv") > attach(data) > str(data) 'data.frame': 105 obs. of 4 variables: \$ Day : Factor w/ 5 levels "Friday","Monday",..: 2 2 2 2 2 2 2 2 2 2 ... \$ Machine : Factor w/ 3 levels "alpha","nu","omega": 1 1 1 1 1 1 1 2 2 2 ... \$ Observation: Factor w/ 7 levels "obs1","obs2",..: 1 2 3 4 5 6 7 1 2 3 ... \$ Time : int 619 440 555 647 582 317 442 677 611 483 ... > summary(aov(Time~(Day*Machine + Error(Observation/(Day*Machine))))) Error: Observation Df Sum Sq Mean Sq F value Pr(>F) Residuals 6 27904 4651 Error: Observation:Day Df Sum Sq Mean Sq F value Pr(>F) Day 4 74074 18518 2.952 0.0408 * Residuals 24 150574 6274 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Error: Observation:Machine Df Sum Sq Mean Sq F value Pr(>F) Machine 2 215990 107995 27.18 3.5e-05 *** Residuals 12 47680 3973 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Error: Observation:Day:Machine Df Sum Sq Mean Sq F value Pr(>F) Day:Machine 8 83544 10443 1.342 0.246 Residuals 48 373468 7781 > t.test(Time[Machine=="alpha"],Time[Machine=="nu"],paired=T) Paired t-test data: Time[Machine == "alpha"] and Time[Machine == "nu"] t = -2.2371, df = 34, p-value = 0.03194 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -85.279662 -4.091767 sample estimates: mean of the differences -44.68571 > t.test(Time[Machine=="alpha"],Time[Machine=="omega"],paired=T) Paired t-test data: Time[Machine == "alpha"] and Time[Machine == "omega"] t = -4.6367, df = 34, p-value = 5.056e-05 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -158.82852 -62.02862 sample estimates: mean of the differences -110.4286 > t.test(Time[Machine=="nu"],Time[Machine=="omega"],paired=T) Paired t-test data: Time[Machine == "nu"] and Time[Machine == "omega"] t = -3.7575, df = 34, p-value = 0.0006451 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -101.30026 -30.18545 sample estimates: mean of the differences -65.74286 > png(filename="/home/sd601/Documents/byday.png") > D = factor(Day, c("Monday","Tuesday","Wednesday","Thursday","Friday")) > plot(Time~D,main="Performance by Day",xlab="Day",ylab="Response time (ms)") > dev.off() png 3 > png(filename="/home/sd601/Documents/bymachine.png") > plot(Time~Machine,main="Performance by Machine",xlab="Machine",ylab="Response time (ms)") > dev.off() png 3