Interaction with Machine Learning
Course structure
This practical course includes 8 2-hour contact sessions. For most sessions, the first hour will be a lecture on a thematic topic related to the course, and the second hour will involve discussion of practical work in progress. The first 2-hour session is dedicated to a practical and theoretical overview of the coruse, and the final 2-hour session is dedicated to presentation of individual research results.
Moodle Link
Note that we do not expect to place any of the course content in Moodle. This link will be used for submission of assignment work. Moodle course: Interaction with Machine Learning 2022-23
Introductory session
11 October: Alan Blackwell and Advait Sarkar - structure of course, overview of HCI, major themes in IUI/TIIS, HCI research methods, planning your study
Timetable for thematic lectures
- 18 October: Mixed initiative interaction (AB)
- 25 October: Labelling as a fundamental problem (AS)
- 1 November: Program synthesis (AB)
- 8 November: Visualisation and visual analytics (AS)
- 15 November: Bias and fairness (AB)
- 22 November: Explainability (Guest lecture from Simone Stumpf, University of Glasgow)
Final session
29 November: Research presentations (all students)
Lecture Slides (will be uploaded after each lecture)
- Lecture 1 - course overview
- Lecture 2 - Mixed initiative
- Lectures 3 & 5 - Labelling and Visualisation
- Lecture 4 - Program synthesis
- Lecture 6 - Fairness and bias
- Lecture 7 - Explanation (Stumpf)
Publication format
Your reports can follow formatting instructions given in the call for papers for the IUI 2021 conference.
Readings
Annotated reading list of relevant publications:
Scanning recent publications at ACM IUI and TIIS will give a useful overview of methods currently used in the field. Archives of both are available on the ACM digital library:
Books:
- The closest to a book of this course (but written for the general public, not as a textbook) is Alan Blackwell's Moral Codes: Designing software without surrender to AI. (MIT Press, 2023/24)
- A more policy-oriented introduction to HCI perspectives on AI is Ben Shneiderman's Human-Centered Artificial Intelligence (Oxford University Press, 2022)
Recent research overviews
- Dudley, J. J., & Kristensson, P. O. (2018). A Review of User Interface Design for Interactive Machine Learning. ACM Transactions on Interactive Intelligent Systems, 8(2), 1–37. https://doi.org/10.1145/3185517
- Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018). Trends and Trajectories for Explainable, Accountable and Intelligible Systems. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, 1–18. https://doi.org/10.1145/3173574.3174156
Background reading, in which senior figures in HCI reflect on the overall themes addressed by this course.
- Ben Shneiderman (2020). Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
- Brad A. Myers and Richard McDaniel (2000). Demonstrational Interfaces: Sometimes You Need a Little Intelligence, Sometimes You Need a Lot
- Alan F. Blackwell (2015). Interacting with an Inferred World: The challenge of machine learning for humane computer interaction.
Candidate replication studies (Part II only)
The following papers address themes that are relevant to the course, and use research methods that are appropriate to the practical component. In most cases, a Part II student would not have access to all the resources (e.g. large sample sizes, large datasets, significant custom software development) to allow full replication of one of these research projects. We therefore expect you to select one part of a study, a reduced number of conditions and/or measurements, and substantially smaller sample sizes. Your research question submission should explain what adaptations you plan to make.
- Predictive Text Encourages Predictable Writing
- The Effect of User Characteristics in Time Series Visualizations
- Recommendation for Video Advertisements based on Personality Traits and Companion Content
- When People and Algorithms Meet: User-reported Problems in Intelligent Everyday Applications
- BigBlueBot: Teaching Strategies for Successful Human-Agent Interactions
- Explanations as Mechanisms for Supporting Algorithmic Transparency
Background in HCI
Teaching materials (including video lectures in 2021) for the
research-oriented Cambridge undergraduate course in Further HCI can be found here:
https://www.cl.cam.ac.uk/teaching/2021/FHCI/
Conduct of research
Before starting your study with human participants, you must make an application to the Computer Lab ethics committee, describing what you plan to do and how you will remove or mitigate any risks.
As described on the application form for ethical review, you should consult the Cambridge Guidance on technology research with human participants, paying particular attention to the type of study you will be conducting, the risks associated with that kind of study, and how you will address them.
Online ethical review form
This form is accessed via the local VPN for computer science department research ("VPN2", not the University-wide VPN provided for other students). All master's students should have installed VPN2 using these instructions. Note that the local server authentication requested by the review form uses your CRSID as username, but the password is your department password (as used for department research facilities such as Kerberos authentication), not your Raven password (as used to access many University-wide IT services).
If master's students are struggling with VPN installation or their department login, send email to the ethics committee secretary including all information requested on the form, as specified in this read-only version of the form. Undergraduate students may send this information directly to AB, who will arrange this on their behalf.
Material from previous years
Much of the content this year will follow the format used for MPhil/Part III module P230 in previous years. Readers wishing to see any of this in advance are welcome to refer to material from previous years.