Federated Learning: Theory and Practice

Objectives

This course aims to extend the machine learning knowledge available to students in Part I (or present in typical undergraduate degrees in other universities), and allow them to understand how these concepts can manifest in a decentralized setting. The course will consider both theoretical (e.g., decentralized optimization) and practical (e.g., networking efficiency) aspects that combine to define this growing area of machine learning. 

At the end of the course students should:

• Understand popular methods used in federated learning
• Be able to construct and scale a simple federated system
• Have gained an appreciation of the core limitations to existing methods, and the approaches available to cope with these issues
• Developed an intuition for related technologies like differential privacy and secure aggregation, and are able to use them within typical federated settings 
• Can reason about the privacy and security issues with federated systems

Lectures

1. Course Overview. Introduction to Federated Learning.
2. Decentralized Optimization.
3. Statistical and Systems Heterogeneity.
4. Variations of Federated Aggregation.
5. Secure Aggregation.
6. Differential Privacy within Federated Systems.
7. Extensions to Federated Analytics.
8. Applications to Speech, Video, Images and Robotics.

Lab sessions

1. Federating a Centralized ML Classifier.
2. Behaviour under Heterogeneity.
3. Scaling a Federated Implementation.
4. Exploring Privacy with Federated Settings

Assessment

Four labs are performed during the course, and students receive 12.5% of their total grade for work done as part of each lab. (For a total of 50% of the total grade from lab work alone). Labs will primarily provide hands-on teaching opportunities, that are then utilized within the lab assignment which is completed outside of the lab contact time. MPhil and Part III students will be given additional questions to answer within their version of the lab assignment which will differ from the assignment given to Part II CST students.

The remainder of the course grade (50%) will be given based on a hands-on project that applies the concepts taught in lectures and labs. This hands-on project will be assessed based on upon a combination of source code, related documentation and brief 8-minute pre-recorded talk that summarizes key project elements (any slides used are also submitted as part of the project). Please note, that in the case of Part II CST students, the talk itself is not examinable -- as such will be made optional to those students.

A range of possible practical projects will be described and offered to students to select from, or alternatively students may propose their own. MPhil and Part III students will select from a project pool that is separate from those offered to Part II CST students. MPhil and Part III projects will contain a greater emphasis on a research element, and the pre-recorded talks provided by this student group will focus on this research contribution. The project will be assessed on the level of student understanding demonstrated, the degree of difficulty, correctness of implementation -- and for Part III/MPhil students the additional criteria of the quality and execution of the research methodology, and depth and quality of results analysis.

This project can be done individually or in groups -- although individual projects will be strongly encouraged. It will be required the project is performed using a code repository that also will contain all documentation -- access to this repository will be shared with course staff (e.g., lecturer and TAs). Where needed, marks assigned to students within a group will be differentiated using this repository as an input. Furthermore if groups are formed, members must be either entirely from Part III/MPhil students or Part II CST, i.e., these two student groups should not mix to form a project group.

Projects will be made available publicly. A maximum word count for written contributions for the project will be enforced. 

Recommended Reading

Readings will be assigned for each lecture. Readings will be taken from either research papers, tutorials, source code or blogs that provide more comprehensive treatment of taught concepts.

Advanced Material sessions - MPhil / Part III Students

These sessions are mandatory for the MPhil and Part III students. Part II CST students may attend if they wish

Topics and announced the first week of class. Selected to extend beyond topics covered in lectures and labs. Content is a mixture of sessions run in the lecture room the are either (1) presentations by guest lectures by an invited domain expert or (2) class-wide discussions regarding one or more related academic papers. Paper discussions will require students to read the paper ahead of the lecture, and a brief discussion primer presentation will be given students before discussions begin.