Advanced Operating Systems

Aims

Systems research refers to the study of a broad range of behaviours arising from complex system design, including: low-level operating systems; resource sharing and scheduling; interactions between hardware and software; network-protocol design and implementation; separation of mutually distrusting parties on a common platform; and control of distributed-system behaviours such as concurrency and data replication. This module will:

1. Teach systems-analysis methodology and practice through tracing and performance profiling experiments;
2. Expose students to real-world systems artefacts such as I/O, IPC,and network-stack implementations, and consider their hardware-software interactions with CPUs;
3. Develop scientific experimentation, analysis and presentation skills through a series of laboratory assignments; and
4. Assign a selection of original research papers to give insight into potential research topics and approaches.

The teaching style will blend lectures and hands-on labs that teach methodology, design principles, and practical skills. Students will be taught about (and assessed via) a series of lab assignments based on practical work. The systems studied are real, and all wires will be live.

Objectives and Learning Goals

On completion of this module, students should:

• Have a good understanding of high-level OS kernel structure
• Gained insight into hardware-software interactions for compute and I/O
• Have practical skills in system tracing and performance analysis
• Have been exposed to research ideas in system structure and behaviour
• Have learned how to perform systems-style performance evaluations
• Have learned how to present systems evaluation results

Syllabus

The sessions are split up into three submodules:

1. Introduction to kernels and kernel tracing/analysis
   The purpose of this submodule is to introduce students to the structure of a contemporary operating system kernel through tracing and profiling.

• Lecture 1: Introduction: OSes, Systems Research, tracing and this course
• Lecture 2: Kernels and Tracing
• Lab 1: Getting Started with Kernel TRacing

2. The Process Model
   This submodule introduces students to concrete implications of the UNIX process model: processes and threads in both userspace and kernelspace, the hardware foundations for kernel and process isolation, system calls, and traps.

• Lecture 3: The Process Model (1)
• Lecture 4: The Process Model (2)
• Lab 3: IPC

3. TCP
   This submodule introduces students to a contemporary network stack, with a particular interest in the TCP protocol. abs will consider both the behaviour of a single TCP connection, exploring the TCP state machine, socket-buffer interactions with flow control, and TCP congestion control. Students will use DUMMYNET to simulate network latency and explore how TCP slow start and congestion avoidance respond to network conditions. The second marked lab assignment will be written.

• Lecture 5: The Network Stack (1)
• Lecture 6: The Network Stack (2)
• Lab 3: TCP

Assessment

Students will complete and submit three lab exercises based on short-form questions and answers. The 'Practice exercise' will be used to develop skills and experience with the teaching platform, tooling, experimental methodology, analysis, and data presentation and contribute 10% of the mark. The remaining two exercises are marked

• Practice Exercise: Getting started with Kernel tracing 10%
• Exercise 2: IPC 40%
• Exercise 3: TCP 50%

Practical work

Five 2-hour in-classroom labs will ask students to develop and use skills in racing and performance analysis as applied to real-world systems artefacts. Results from these labs (and follow-up work by students outside of the classroom) will be the primary input to the lab exercises.

Typical labs will involve using tracing and profiling to characterise specific behaviours (e.g., file I/O in terms of system calls and traps) as well as perform root-cause analysis of application-level behaviours (e.g., exploring network clients and servers to better understand real-world TCP behaviour).

Students will be required to attend three separately scheduled ten-minute meetings with the instructor to review their submissions over the course of the term. These meetings will take place before final marks are assigned for lab exercise, and will give students the opportunity to explain their approach, results, and insights.

The use of AI coding assistants in generating programs, scripts, and instrumentation for data collection is permitted in this module. AI may not be used in data analysis, writing, or data presentation, as these are essential skills developed as part of this module.

Recommended Reading

Primary module texts

Course texts provide instruction on statistics, operating-system design and implementation, and system tracing. You will be asked to read selected chapters from these, but will likely find other content in them useful as you proceed with the labs.

Marshall Kirk McKusick, George V. Neville-Neil, and Robert N. M. Watson. The Design and Implementation of the FreeBSD Operating System, 2nd Edition, Pearson Education, Boston, MA, USA, September 2014.

Brendan Gregg and Jim Mauro. DTrace: Dynamic Tracing in Oracle Solaris, Mac OS X and FreeBSD, Prentice Hall Press, Upper Saddle River, NJ, USA, April 2011.

Additional texts

Abraham Silberschatz, Peter Baer Galvin, and Greg Gagne, Operating System Concepts, Eighth Edition, John Wiley and Sons, Inc., New York, NY, USA, July 2008.

Brendan Gregg. Systems Performance: Enterprise and the Cloud, Prentice Hall Press, Upper Saddle River, NJ, USA, October 2013.

Research-paper readings

Research-paper readings will be announced as the terms proceed, but will likely include original papers on BPF, DTrace, OS scheduling, OS scalability, network stacks, and systems modelling.