Principles of Machine Learning Systems
Principal lecturer: Dr Nic Lane
Additional lecturer: Dr Pedro Porto Buarque de Gusmao
Taken by: MPhil ACS, Part III
Code: L46
Term: Michaelmas
Hours: 16 (8 × two-hour sessions)
Class limit: max. 30 students
Prerequisites: It is recommended that students have successfully completed introductory courses, at an undergraduate level, in: 1) operating systems, 2) computer architecture, and 3) machine learning. In addition, this course will heavily focus on deep neural network methodologies and assume familiarity with common neural architectures and related algorithms. If these topics were not covered within the machine learning course taken, students should supplement by reviewing material in a book like: "Dive into Deep Learning". Finally, students are assumed to be comfortable with programming in Python.
Moodle, timetable
Aims
This course will examine the emerging principles and methodologies that underpin scalable and efficient machine learning systems. Primarily, the course will focus on an exciting cross-section of algorithms and system techniques that are used to support the training and inference of machine learning models under a spectrum of computing systems that range from constrained embedded systems up to large-scale distributed systems. It will also touch upon the new engineering practices that are developing in support of such systems at scale. When needed to appreciate issues of scalability and efficiency, the course will drill down to certain aspects of computer architecture, systems software and distributed systems and explore how these interact with the usage and deployment of state-of-the-art machine learning.
Syllabus
Topics covered may include the following, with confirmation a month before the course begins:
- System Performance Trade-offs
- Distributed Learning Algorithms
- Model Compression
- Deep Learning Compilers
- Frameworks and Run-times
- Scalable Inference Serving
- Development Practices
- Automated Machine Learning
- Federated Learning
Primarily, topics are covered with conventional lectures. However, where appropriate, material will be delivered through hands-on lab tutorials. Lab tutorials will make use of hardware including ARM microcontrollers and multi-GPU machines to explore forms of efficient machine learning (any necessary equipment will be provided to students)
Assessment
Each student will be assessed on 3 labs which will be worth 30% of their grade. They will also undertake a written project report which will be worth 70% of the grade. This report will detail an investigation into a particular aspect of machine learning systems, this report will be made available publicly.
Further Information
Due to infectious respiratory diseases, the method of teaching for this module may be adjusted to cater for physical distancing and students who are working remotely. Unless otherwise advised, this module will be taught in person.