Interaction with Machine Learning

Course structure

This practical course includes 8 2-hour contact sessions. For most sessions, the first hour will be a lecture on a thematic topic related to the course, and the second hour will involve discussion of practical work in progress. The course is run in parallel with a Cambridge Digital Humanities group project. CDH students will have separate practical sessions, but will attend lectures and participate in discussion of individual research. The first 2-hour session is dedicated to a practical and theoretical overview of the coruse, and the final 2-hour session is dedicated to presentation of individual research results.

Moodle Link

Note that we do not expect to place any of the course content in Moodle. This link will be used for submission of assignment work. It can also be used to access a Zoom link for use by students who are unable to attend a lecture through illness. (Note that recordings will not be published, and discussion of practical work will not be streamed).
Moodle course: Interaction with Machine Learning 2021-22

Lecture Slides

• Lecture 1 - course overview
• Lectures 2 & 3 - Labelling and Visualisation
• Lecture 4 - Mixed initiative
• Lecture 5 - Program synthesis
• Lecture 6 - Explanation (Stumpf)
• Lecture 7 - Fairness and bias

Introductory session

25 January: Alan Blackwell and Advait Sarkar - structure of course, overview of HCI, major themes in IUI/TIIS, HCI research methods, planning your study

Timetable for thematic lectures

• 1 February: Labelling as a fundamental problem (AS)
• 8 February: Visualisation and visual analytics (AS)
• 15 February: Mixed initiative interaction (AB)
• 22 February: Program synthesis (AB)
• 1 March: Explainability (Guest lecture from Simone Stumpf, University of Glasgow)
• 8 March: Bias and fairness (AB)

Final session

15 March: Research presentations (all students)

Publication format

Your reports can follow formatting instructions given in the call for papers for the IUI 2021 conference.

Readings

Annotated reading list of relevant publications:

• IWML-2022-reading-list.pdf

Scanning recent publications at ACM IUI and TIIS will give a useful overview of methods currently used in the field. Archives of both are available on the ACM digital library:

• Archives of ACM IUI
• Archives of ACM TIIS

Recent research overviews

• Dudley, J. J., & Kristensson, P. O. (2018). A Review of User Interface Design for Interactive Machine Learning. ACM Transactions on Interactive Intelligent Systems, 8(2), 1–37. https://doi.org/10.1145/3185517
• Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018). Trends and Trajectories for Explainable, Accountable and Intelligible Systems. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, 1–18. https://doi.org/10.1145/3173574.3174156

Background reading, in which senior figures in HCI reflect on the overall themes addressed by this course.

• Ben Shneiderman (2020). Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
• Brad A. Myers and Richard McDaniel (2000). Demonstrational Interfaces: Sometimes You Need a Little Intelligence, Sometimes You Need a Lot
• Alan F. Blackwell (2015). Interacting with an Inferred World: The challenge of machine learning for humane computer interaction.

Candidate replication studies (Part II only)

The following papers address themes that are relevant to the course, and use research methods that are appropriate to the practical component. In most cases, a Part II student would not have access to all the resources (e.g. large sample sizes, large datasets, significant custom software development) to allow full replication of one of these research projects. We therefore expect you to select one part of a study, a reduced number of conditions and/or measurements, and substantially smaller sample sizes. Your research question submission should explain what adaptations you plan to make.

• Predictive Text Encourages Predictable Writing
• The Effect of User Characteristics in Time Series Visualizations
• Recommendation for Video Advertisements based on Personality Traits and Companion Content
• When People and Algorithms Meet: User-reported Problems in Intelligent Everyday Applications
• BigBlueBot: Teaching Strategies for Successful Human-Agent Interactions
• Explanations as Mechanisms for Supporting Algorithmic Transparency

Background in HCI

Teaching materials (including video lectures from 2021) for the research-oriented Cambridge undergraduate course in HCI can be found here:
https://www.cl.cam.ac.uk/teaching/2021/FHCI/

Conduct of research

Before starting your study with human participants, you must make an application to the Computer Lab ethics committee, describing what you plan to do and how you will remove or mitigate any risks.

Online review form
(Note that access to this form requires login to department VPN. Undergraduates please prepare answers using the read-only version, then ask one of the lecturers to submit on your behalf.)

As described on the form, you should also consult the Cambridge Guidance on technology research with human participants, paying particular attention to the type of study you will be conducting, the risks associated with that kind of study, and how you will address them.

Material from previous years

Much of the content this year will follow the format used for MPhil/Part III module P230 in previous years. Readers wishing to see any of this in advance are welcome to refer to material from previous years.