Practical Research in Human-centred AI

Students will carry out a practical experimental study, replicating a piece of published research.

This work will be completed incrementally through the term, with feedback on each section providing opportunity for improvement before submission and assessment of a final report. These six incremental submissions will together contribute 20% to the final module mark.

The six phased submissions should address the following aspects of the empirical study:

1. Due at noon on 16 October: Selection of study for replication, with summary of the key research question and any necessary changes of scope or other adaptations
2. Due at noon on 23 October: Detailed work plan for data collection and analysis
3. Due at noon on 6 November: Literature review including summary of the original publication, and other work that cites it
4. Due at noon on 13 November: Introduction to the replicated study
5. Due at noon on 20 November: Results and data analysis
6. Due at noon on 27 November: Discussion of results and conclusion

The final report is due at noon on 1 December

The six phased submissions will be pass/fail rather than graded exercises: i.e., for each submission, 100% of the mark is awarded for satisfactory completion and 0% for inadequate work or failure to submit. Feedback on each phased submission will include an indicative mark to guide preparation of the final report.

Formatting

For the phased assignments, please submit to Moodle as PDF if possible. Please put your name at the top, followed by "(Part II)"

For each phased assignment, please include all the text of the previous phases within the same document, but with the new text that has been prepared for this phase clearly emphasised (e.g. in a different colour). Where you have decided to revise the text from a previous phase in response to feedback, please also mark the revised text, if you would like to receive further feedback on it.

Formatting of final report

The final report is to be formatted as a conference submission to an ACM conference such as IUI or CHI. (see Materials tab for links).

The final assignment will be a written report of the research findings. It is allowable to include text from the earlier phased submissions, with updates as needed in response to the feedback given. The report will be in the style of a conference submission. You should pay particular attention to overall length, remembering that student projects would typically be submitted to a conference as a short papers. Note that while guidance for draft text and course workload has indicated approximate numbers of words, ACM conference submisssions are determined as number of pages. Precise word count is not a priority, but there is no expectation that you should write more than 3,000 words. Marks will be based on the quality of the work, and appropriateness to the contribution being made, rather than the length of the paper. The final assignment will contribute 80% of the final mark.

Notes on presentation of a replication study

The models of previous research discussed during the course are not usually replication studies, which may lead you to wonder how you can write up your own work in the form of a research contribution. Here are a few hints:

• In the introduction, state clearly that your work is based on (replicates, challenges, extends ...) a previous publication, and cite that publication.
• In the background / literature review, it will be necessary to briefly summarise the nature and contribution of the original study. You may need to refer to some of the literature cited in the original study.
• Your literature review is also likely to include references to further publications that you have consulted. In particular, if you are challenging or extending their work, you will need to set out the technical or theoretical basis on which you are doing this.
• The description of your research methods should not waste too much space repeating material from the original paper, although you will need to give enough information that readers can follow what you are doing as a standalone piece of work. Where necessary, cite the original paper for further detail.
• You will need to pay more attention to any changes you are making to the original, explaining why this has been done, and hopefully why it will be of interest to the reader.
• In the description of results and analysis, the most important thing is to make comparisons to the original paper. Any differences that you observed will be of particular interest, especially if they might lead to a challenge or alternative interpretation of the earlier work.
• In the conclusion of your paper, you need to explain, with justification from your results, what you have achieved. This may be to confirm the original findings, which is a useful replication. You could explain why confirmation is important and significant. If you are challenging some aspect of the original findings, you need to make your case persuasively, and also explain the implications. If you are extending or applying the original results, this will be a new contribution of your own, which can be presented in the style of any other original research paper.

Although not required for Part II Units, ACS students also compile a reflective diary through the term, reporting on discussion in the weekly sessions. Part II students are invited to submit their own diary if they wish to do so. Feedback will be given, but this will not contribute to the course mark. Diary entries include citations to key references, notes of possible further reading, summary of key points, questions relevant to the personal project, and points of interest noted in relation to the work of other students.