Probabilistic Machine Learning

Aim

Teach the probabilistic basis of advanced techniques in neural network modelling

Syllabus

Deep learning as “probabilistic modelling with ultra-high dimensional models”. It covers:

(1) supervised learning and generative modelling as maximum likelihood fitting,

(2) variational, adversarial, and energy-based bounds for fitting distributions,

(3) approaches to generalization / inference, in particular, approaches to modelling uncertainty,

(4) causal deep learning,

(5) some standard models / modalities, including transformers and their link with Gaussian processes.

The course will describe this material in the style of “applied probability modelling”, but it will also emphasize the practical aspect of running these models on actual real datasets.

Objectives

Equip students to understand and apply advanced theory in deep learning

Assessment

Assessment and marking

The module is assessed through three components:

1. Written responses to research papers: 20%

Students will complete four short written responses, each worth 5% of the overall module mark.

For four of the eight lectures, students will be given a research paper to read and one or more targeted questions to answer. Each response should be no longer than half a page.

The questions will assess students' ability to interpret and engage critically with the paper. For example, students may be asked to:

• identify an unstated assumption or premise;
• explain an argument that is implied rather than directly stated; or
• rewrite equations using the language and concepts introduced in the module.

Each response will be marked separately.

Confirmation of authorship

During the term, each student will be asked to talk through at least one of their written responses. This may take place:

• during a lecture, through a short explanation to the class; or
• in a brief individual viva following a lecture.

Students will be selected at random. Students who do not wish to speak in front of the class may request an individual viva instead.

The purpose of this discussion is to confirm authorship and the student's understanding of the submitted work. If concerns about authorship remain following the discussion, the matter may be referred to the Examiners for further consideration.

2. Assessed exercises: 30%

Students will complete two assessed exercises, each worth 15% of the overall module mark.

Each exercise will be approximately 800 words and may include a coding element. Marks will primarily be awarded for:

• interpretation;
• explanation;
• critical reasoning; and
• understanding of the methods and results.

The quality of the code alone will not determine the mark.

Confirmation of authorship

Each student will attend a short individual viva relating to one of the two exercises. Students will be selected at random to complete their viva after either the first or second exercise.

The viva will be used to confirm the student's authorship and understanding of the submitted work. If concerns remain following the viva, the matter may be referred to the Examiners.

3. Research proposal and report: 50%

This component consists of:

• a one-page research proposal, worth 5%; and
• a research report, worth 45%.

The research proposal will be submitted at the end of term. All students will have an opportunity to discuss their proposed research with the module convenor before the vacation. There will be no viva for the proposal.

The research report will be completed during the vacation and should be approximately 4,000 words, excluding any accompanying code.

After submitting the report, every student will attend an individual viva. The purpose of the viva is to confirm authorship and the student's understanding of the research, methods, code, findings and conclusions presented in the report.

If concerns about authorship remain following the viva, the matter may be referred to the Examiners.

Assessment summary

• Four written paper responses: 20%
• Two assessed exercises: 30%
• Research proposal: 5%
• Research report: 45%
• Total: 100%

Use of generative AI

Students must follow the module and University guidance on the permitted use of generative AI. Students should retain appropriate notes, drafts, code versions and other evidence of how their work was developed, as they may be asked to explain their approach during a discussion or viva.

Recommended reading