Computer Laboratory

Course pages 2015–16

Usability of Programming Languages

Principal lecturer: Prof Alan Blackwell
Taken by: MPhil ACS, Part III
Code: P201
Hours: 16 (4 x introductory lectures, 4 × two-hour seminar sessions, 2 x demonstrated experimental sessions)
Class limit: 15 students
Prerequisites: Introductory HCI course presumed, but not essential


Compiler construction is one of the basic skills of all computer scientists, and thousands of new programming, scripting and customisation languages are created every year. Yet very few of these succeed in the market, or are well regarded by their users. This course addresses the research questions underlying the success of new programmable tools. A programming language is essentially a means of communicating between humans and computers. Traditional computer science research has studied the machine end of the communications link at great length, but there is a shortage of knowledge and research methods for understanding the human end of the link. This course provides practical research skills necessary to make advances in this essential field. The skills acquired will also be valuable for students intending to pursue research in advanced HCI, or designing evaluation studies as a part of their MPhil research project.


  • Lecture 1: Principles of human factors in programming. Cognitive models of the programming process, introduction to the leading world research centres and current areas of theoretical attention. Integration of programming within the software development process. Sources of individual variation. Relationship between syntax and tools, including integrated development environments, visual languages, spreadsheets, scripting.
  • Lecture 2: Research methods in the study of programming. Controlled experimental methods, typical tasks, experimental manipulations of programming tools. Measurement, observation and protocol analysis techniques. Experiment design. Field study methods.
  • Lecture 3: Special classes of programming language use, including end-user programming, mashup and composition tools, educational languages. Overview of classes of technical approach in novel language development, requirements capture and evaluation for these classes.
  • Lecture 4: Planning practical empirical studies. Students will identify specific programming languages for investigation, and representative tasks and measures suitable for usability evaluation. Candidate languages can be drawn from current research in the Computer Laboratory or at Microsoft Research Cambridge, recent product launches and prototypes, or languages implemented in project work by MPhil students themselves.
  • Experiments designed and conducted by students will be presented in seminar sessions, for discussion and feedback among the class. Seminars will also include invited guests, postdoctoral researchers and/or PhD students carrying out active research in programming language design and evaluation.
  • Seminar 1 and 2: presentation and feedback on study proposals
  • Seminar 3 and 4: presentation of research study findings


On completion of this practically-oriented module, students should:

  • Be able to work in stages through the development of research hypotheses and proposals, to the design and implementation of their own empirical study of languages or programmable systems.
  • Have acquired the theory base and research skills that are necessary for work in this field, through practical experimental work.
  • Have reached a level of competence sufficient for conducting and publishing their own original research in the field.


Students will prepare three pieces of written work during the course. Assignments B and C will be submitted in two iterations - an initial proposal, then a revised version based on feedback in seminars and from the course director.

  • Assignment A: background to a proposed study, including description of the target language, paradigm, tool or environment, a review of the relevant theoretical literature and previous empirical studies
  • Assignment B: structure of the experimental design, detailed protocol of the proposed study, and outline of analytic methods to be used.
  • Assignment C: full experimental report, building on final versions of assignments A and B, and presenting data analysis and findings in a format suitable for publication at a specialist research venue such as the psychology of programming interest group.

Practical work

Practical sessions will make use of the dedicated experimental observation facility in the Gates Building, which provides a range of recording and measurement equipment. Each student will design and conduct their own experiment, spread over approximately four practical sessions. Two practical sessions will have a demonstrator present. Students will also be required to participate in at least three experiments conducted by other members of the class.


  • Assignment A, 10%
  • Assignment B outline version, 10%
  • Assignment B detailed proposal, 15%
  • Assignment C outline version, 10%
  • Assignment C full report, 35%
  • Contributions made in four seminars, 2.5% each
  • Final presentation 10%

Recommended reading

Online proceedings of the Psychology of Programming Interest Group
Cambridge guidance for human participants in technology research
Cairns, P. and Cox, A.L. (2008) Research Methods for Human-Computer Interaction. Cambridge University Press.
Hoc, J.M, Green , T.R.G, Samurcay, R and Gilmore, D.J (Eds.) (1990) Psychology of Programming. Academic Press.
Carroll, J.M. (Ed) (2003). HCI Models, Theories and Frameworks: Toward a multidisciplinary science. Morgan Kaufmann.