Computer Laboratory

Large-Scale Data Processing and Optimisation (2026-2027 Michaelmas Term)

LSDPO - R244

review_log

Open Source Projects

Reading Club Papers

Mini-Project List

Contact

 

 

 

 

 

 

 

  

 

Overview

This module provides an introduction to large-scale data processing, optimisation, and the impact on computer system's architecture. Large-scale distributed applications with high volume data processing will grow ever more in importance. Supporting the design and implementation of robust, secure, and heterogeneous large-scale distributed systems is essential. To deal with distributed systems with a large and complex parameter space, tuning and optimising computer systems is becoming an important and complex task, which also deals with the characteristics of input data and algorithms used in the applications. Algorithm designers are often unaware of the constraints imposed by systems and the best way to consider these when designing algorithms with massive volume of data. On the other hand, computer systems often miss advances in algorithm design that can be used to cut down processing time and scale up systems in terms of the size of the problem they can address. Integrating machine learning approaches (e.g. Bayesian Optimisation, Reinforcement Learning) for system optimisation and Large Language Model (LLM) will also be explored in this course.

Recent computer systems have a massive task to enable heavy data processing for fast training and inference, e.g. dealing with LLM. This demands more sophisticated system architecture, advanced hardware, and fast tensor operation capable compilers. Complex optimisation plays a crucial role here and it is fundamental at various stages of the development.

On completion of this module, the students should:

  • Understand key concepts of scalable data processing approaches in future computer systems
  • Obtain a clear understanding of building distributed systems using data centric programming and large-scale data processing
  • Understand a large and complex parameter space in computer system's optimisation and applicability of Machine Learning approach

Module Structure

The module consists of 8 sessions, with 5 sessions on specific aspects of large-scale data processing research. Each session discusses 3-4 papers, led by the assigned students. One session is a hands-on tutorial on dataflow programming fundamentals. The first session advises on how to read/review a paper together with a brief introduction on different perspectives in large-scale data processing and optimisation. The last session is dedicated to the student presentation of opensource project studies. 

Schedule and Reading List

All the sessions will be in the classroom (FW26). We will meet every Wednesday (from October 14 to December 2) in 2026. The time slot is 10:00-12:00.1

 

 2026/10/14 Session 1: Introduction to Large-Scale Data Processing and Optimisation

 

Introduction to R244 (slides: Course Guide)

·       Assignment details

·       Guidance of how to read/review/present a paper

·       Guidance to Open Source Mini Project

 

 Overview of technologies for a large scale data processing and optimisation (slides: Overview)

 

 2026/10/21 Session 2: Graph Processing/ Search Space & Optimisation

 

Optimisation can be viewed as a search problem over a parameter space. This session focuses on two key aspects:

·       Graph-Based Search Space Representation: Graphs as powerful representations of complex search spaces, including scalable processing of graph-structured data and computational models.

·       Search Space Exploration and Optimisation: Methods and algorithms for efficiently exploring search spaces and identifying optimal or near-optimal solutions.

 

1.       S. Liu et al.: EvoX: Meta-Evolution for Automated Discovery, arXiv, 2026.

2.       G. Liao et al.: KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta, arXiv, 2026.

3.       Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, J. Hellerstein: Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud, VLDB, 2012.

4.       J. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin: Powergraph: distributed graph-parallel computation on natural graphs. OSDI, 2012.

5.       J. Shun and G. Blelloch: Ligra: A Lightweight Graph Processing Framework for Shared Memory, PPoPP, 2013. 

6.       J.  Gonzalez, R. Xin, A. Dave, D. Crankshaw, M. Franklin, I. Stoica: GraphX: Graph Processing in a Distributed Dataflow Framework, OSDI, 2014. 

7.       A. Kyrola and G. Blelloch: Graphchi: Large-scale graph computation on just a PC, OSDI, 2012.  

8.       A. Roy, I. Mihailovic, W. Zwaenepoel:   X-Stream: Edge-Centric Graph Processing using Streaming Partitions, SOSP, 2013.

9.       A. Roy, L. Bindschaedler, J. Malicevic and W. Zwaenepoel: Chaos: Scale-out Graph Processing from Secondary Storage , SOSP, 2015.

10.    F. McSherry, M. Isard and D. Murray: Scalability! But at what COST? , HOTOS, 2015.

11.    Z. Jia, Y. Kwon, G. Shipman, P. McCormick, M. Erez, A. Aiken: A Distributed Multi-GPU System for Fast Graph Processing, VLDB, 2018.

12.    K. Nilakant, V. Dalibard, A. Roy, and E. Yoneki: PrefEdge: SSD Prefetcher for Large-Scale Graph Traversal.  ACM International Systems and Storage Conference (SYSTOR), 2014.

13.    L. Bindschaedler, J. Malicevic, N. Schiper, A. Goel, W. Zwaenepoel: Rock you like a Hurricane: taming skew in large scale analytics. EuroSys, 2018.

14.    J. Ansel et al. Opentuner: an extensible framework for program autotuning. PACT, 2014.

15.    J. Ansel et al.: Petabricks: A language and compiler for algorithmic choice, PLDI, 2009.

16.    O. Alipourfard et al.: CherryPick: Adaptively Unearthing the Best Cloud Configurations for Big Data Analytics, NSDI, 2017.

17.    Cheng et al.: Barbarians at the gate: How ai is upending systems research, arXiv, 2025.

18.    Cheng et al.: Let the barbarians in: How ai can accelerate systems performance research, arXiv, 2025.

19.    Y. Yang, et al.: Equality Saturation for Tensor Graph Superoptimization, MLSys, 2021.  

20.    J. Hartmann, et al.: Optimizing Tensor Computation Graphs with Equality Saturation and Monte Carlo Tree Search, PACT, 2024.

21.    G. He, et al.:  X-RLflow: Graph Reinforcement Learning for Neural Network Subgraphs Transformation, MLSys, 2023.

22.    G. He, et al.: MCTS-GEB: Monte Carlo Tree Search is a Good E-graph Builder, EuroMLSys, 2023.

 

 2026/10/28 Session 3: Data flow programming

 

Data flow programming and Cluster Computing are essential for a large scale data processing. In ML, dataflow programming holds the key to natural, modular, streamlined ML specification integrated with pre- and post-processing and covering typical ML needs.

 

1.       Yuan Yu, Michael Isard, D. Fetterly, M. Budiu, U. Erlingsson, P.K. Gunda, J. Currey: DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language, OSDI, 2008.

2.       G. Malewicz, M. Austern, A. Bik, J. Dehnert, I. Horn, N. Leiser, and G. Czajkowski: Pregel: A System for Large-Scale Graph Processing, SIGMOD, 2010.

3. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. Franklin, S. Shenker, I. Stoica: Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing, NSDI, 2013.

4. J. Gjengset, M. Schwarzkopf, J. Behrens, L. T. Araujo, M. Ek, E. Kohler, M. F. Kaashoek and R. Morris: Noria: dynamic, partially-stateful data-flowfor high-performance web applications, OSDI 2018.

5. J. Dean, S. Ghemawat: MapReduce: Simplified Data Processing on Large Clusters, OSDI, 2004.

6. Derek Murray, Malte Schwarzkopf, Christopher Smowton, Steven Smith, Anil Madhavapeddy and Steven Hand: Ciel: a universal execution engine for distributed data-flow computing, NSDI 2011. 

7. Naiad Frank McSherry's Talk on Differential Dataflow is here.

7.1. Frank McSherry, Rebecca Isaacs, Michael Isard, and Derek G. Murray: Composable Incremental and Iterative Data-Parallel Computation with Naiad, no. MSR-TR-2012-105, 2012. 

7.2. D. Murray, F. McSherry, R. Isaacs, M. Isard, P. Barham, M. Abadi: Naiad: A Timely Dataflow System, SOSP, 2013. 

8. P. Bhatotia, A. Wieder, R. Rodrigues, U. A. Acar, and R. Pasquini: Incoop: MapReduce for incremental computation, ACM SOCC, 2011.

9. M. Abadi et al. Tensorflow: A system for large-scale machine learning. OSDI, 2016.

9.1   M. Abadi et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems, Preliminary White Paper, 2015.

9.2. M. Abadi, M. Isard and D. Murray: A Computational Model for TensorFlow - An Introduction, MAPL, 2017.

10. M. Looks et al.: Deep Learning with Dynamic Computation Graphs, ICLR, 2017.

11. Y. Yu et al.: Dynamic Control Flow in Large-Scale Machine Learning, EuroSys, 2017.

12. R. Nishihara, P. Moritz, et al.: Ray:A Distributed Framework for Emerging AI Applications, OSDI, 2018.

13. M. Schaarschmidt, S. Mika, K. Fricke, E. Yoneki: RLgraph: Flexible Computation Graphs for Deep Reinforcement Learning, SysML, 2019.

14. S. Li, Y. Zhao, R. Varma, et. al: PyTorch Distributed: Experiences on Accelerating Data Parallel Training, VLDB, 2020.

15. T. Lévai, F. Németh, and G. Rétvári: Batchy Batch scheduling Data Flow Graphs with Service level Objective, NSDI, 2020.

16. P. Barham, et al.: Pathways: Asynchronous Distributed Dataflow for ML, MLSys, 2022.  

17. L. Zheng, et al.: Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning, OSDI, 2022.  

18. H. Zhu, et al.: MSRL: Distributed Reinforcement Learning with Dataflow Fragments, USENIX_ATC, 2023.  

19. A. Sarker, et al.: AAFLOW: Scalable Patterns for Agentic AI Workflows, arXiv, 2026.

20. D. Gao, et al.: SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters, HPDC, 2026.

21. D. Gao, et al.: AgentScope: A Flexible yet Robust Multi-Agent Platform, ArXiv, 2024.

22. M. Balandat et al.: BOTORCH: Bayesian Optimization in PyTorch, Arxiv 2020. 

 

 2026/11/04 Session 4: Data Flow Programming Tutorial  

 

Data Flow Programming Tutorial Hands-on tutorial session of distributed Data Flow Programming.

 

Tutor: Sami Alabed (Prometheus/Google DeepMind) and assistant Youhe Jiang.

 

·        Be familiar to use Python.

·        Linux is preferable, but any OS in the laptop should be fine!

·        This tutorial session will be in FW26. You will use your own laptop.

 

Past year's tutorial sessions: 

·        2024: Data Flow Programming Tutorial
2021: Dataflow programming using TensorFlow
2020: Dataflow programming using TensorFlow
2019: Dataflow programming using TensorFlow
2015: Naiad Tutorial
2014: Dataflow programming using CIEL

 

  2026/11/11 Session 5: Probabilistic Programming 

 

Guest lecture: TBC.

 

Reading Club @10:00.

 

1.       E. Bingham et al.: Pyro: Deep Universal Probabilistic Programming, Journal of Machine Learning Research, 2019. 

2.       D. Tran et al.: Edward: A library for probabilistic modeling, inference, and criticism, arXiv, 2017.

3.       F. Wood, J. van de Meent, V. Mansinghka: A new approach to probabilistic programming inference, AISTATS, 2014.

4.       B. Paige and F. Wood: A compilation target for probabilistic programming languages, ICML, 2014.

5.       J. Ai et al.: HackPPL: a universal probabilistic programming language, MAPL, 2019.

6.       V. Dalibard, M. Schaarschmidt, and E. Yoneki: BOAT: Building Auto-Tuners with Structured Bayesian Optimization, WWW, 2017. 

7.       W. Neiswanger et al.: ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language, Arxiv, 2019. 

8.       J. Shao, et al.: Tensor Program Optimization with Probabilistic Programs, NeurIPS, 2022. 

9.    Maraval, A. et al.: Sample-Efficient Optimisation with Probabilistic Transformer Surrogates, ArXiv, 2022.

10.    J. Snoek, H. Larochelle, and R. Adams: Practical Bayesian Optimization of Machine Learning Algorithms, NIPS, 2012.

11.    A. Klein, S. Falkner, S. Bartels, P. Hennig, F. Hutter: Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets, AISTAS, 2017.

12.    R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. Gonzalez, I. Stoica: Tune: A Research Platform for Distributed Model Selection and Training, ICML, 2018.  

13.    Z. Wang, C. Gehring, P. Kohli, and S. Jegelka: Batched Large-scale Bayesian Optimization in High-dimensional Spaces, AISTATS, 2018.  

14.    A. Grosnit et al.: High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning, arxiv, 2021.  

15.    C. Lin, J. Miano, and E. Dyer: Bayesian optimization for modular black-box systems with switching costs, UAI, 2021.  

16.    E. H. Lee, D. Eriksson, V. Perrone and M. Seeger: A Nonmyopic Approach to Cost-Constrained Bayesian Optimization, UAI, 2021.  

17.    G. Claret, et al.: Bayesian Inference using Data Flow Analysis, ESEC/FSE, 2013.  

18.    A. Lew, et al.: Probabilistic Programming with Stochastic Probabilities, PLDI, 2023.  

19.   L. Mandel, et al.: PPDL: LLM-Based Flows as Probabilistic Programs, PMLR, 2026.

20.   A. Lew, et al.: Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs, ICML, 2023.

21.   Y. Jiang, et al.: BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization, MLSys, 2026.

 

 2026/11/18 Session 6: Optimisations in Computer Systems 

 

Many faces of Optimisations in Computer Systems with ML (e.g. AutoTuner, Automatic Parallelism, Database, Devie Placement/Scheduling).


1.1 A. Mirhoseini et al.: Device Placement Optimization with Reinforcement Learning, ICML, 2017.
1.2. A. Mirhoseini, A. Goldie, H. Pham, B. Steiner, Q. Le and J. Dean: A Hierarchical Mode for Device Placement, ICLR, 2018.  

2.1 A. Mirhoseini, A. Goldie, et al.: A graph placement methodology for fast chip design, Nature, 2021.  
2.2 A. Mirhoseini, A. Goldie, et al.: Chip Placement with Deep Reinforcement Learning, ISPD, 2020.  

3. R. Addanki, S. B. Venkatakrishnan, S. Gupta, H. Mao, M. Alizadeh: Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning, arXiv, 2019. 

4. R. Marcus, P. Negi, Parimarjan, H. Mao, C. Zhang, M. Alizadeh, T. Kraska, O. Papaemmanouil, and N. Tatbul: Neo: A Learned Query Optimizer, VLDB, 2019. 

6. A. Pavlo et al.: Self-Driving Database Management Systems, CIDR, 2017.
7. H. Mao et al.: Park: An Open Platform for Learning-Augmented Computer Systems, OpenReview, 2019.

9. G. Li, X. Zhou, S. Li, and B. Gao: Qtune: RL for DB query optimisation, VLDB, 2019.

10. D Van Aken, A Pavlo, GJ Gordon, and B Zhang: Automatic database management system tuning through large-scale machine learning, SIGMOD, 2017.

11. D. Aken , D. Yang, S. Brillard, A. Fiorino, B. Zhang, C. Bilien, and A. Pavlo: An Inquiry into Machine Learning-based Automatic Configuration Tuning Services on Real-World Database Management Systems, VLDB, 2021.

12. Zhao, Y. et al.: TOD: GPU-accelerated Outlier Detection via Tensor Operations, VLDB, 2022.

13. Kraska, T., Alizadeh, M., Beutel, A., Chi, E.H., Ding, J., Kristo, A., Leclerc, G., Madden, S., Mao, H. and Nathan, V.: SageDB: A learned database system, CIDR, 2019. 

14. D. Ha and J. Schmidhuber: World Models, arXiv, 2018 (https://worldmodels.github.io). 

15. E. Liang, Z. Wu, M. Luo, S. Mika, J. Gonzalez, and I. Stoica: RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problem, NeurIPS, 2021.  

16. S. Cereda, S. Valladares, P. Cremonesi and S. Doni: CGPTuner: a Contextual Gaussian Process Bandit Approach for the Automatic Tuning of IT Configurations Under Varying Workload Conditions, VLDB, 2021.

17. Z. Jia, et al.: Unity: Accelerating DNN Training Through Joint Optimization of Algebraic Transformations and Parallelization, OSDI, 2022.  

18. J. Xing, et al.: Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance, MLSYS, 2022.  

19. M. Lindauer, et al.: SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization, Journal of Machine Learning Research, JMLR, 2021.  

20. M. Wagenländer, et al.: Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections, SOSP, 2024.  

21. X. Miao, et al.: FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning, arXiv, 2024.  

22. Y. Mei, et al.: Helix: Distributed Serving of Large Language Models via Max-Flow on Heterogeneous GPUs, arXiv, 2024.  

23. J. Juravsky, et al.: Hydragen: High-Throughput LLM Inference with Shared Prefixes, arXiv, 2024.  

24. H. Zhang, et al.: LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference, ISCA, 2024.  

25. J. Cheng, et al.: A Dataflow Compiler for Efficient LLM Inference using Custom Microscaling Formats, arXiv, 2023.  

26. Y. Zhong, et al.: DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving, OSDI, 2024.  

27. W. Kwon, et al.: Efficient Memory Management for Large Language Model Serving with PagedAttention, SOSP, 2023.  

28.  Y. Jiang, et al.: ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments, MLSYS, 2025.  

29. S. Tiwari et al.: CacheWise Combines prefix-aware scheduling and predictive eviction to improve KV-cache reuse for coding agents, arXiv, 2026.

30. H. Kang, et al.: ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System, arXiv, 2026.

31. T. Xia, et al.: Idleness is Relative: Exploiting Tool-Call Idle Windows for Offloading in Agentic Systems with MORI, arXiv, 2026.

32. J. Wang, et al.: SMetricUses session-centric request routing to balance serving load while preserving KV-cache locality for agentic workloads, arXiv, 2026.

33. R. Qin, et al.:  Mooncake: Trading More Storage for Less Computation — A KVCache-centric Architecture for Serving LLM Chatbot, FAST, 2025.

34. A. Ouyang, et al.: KernelBench: Can LLMs Write Efficient GPU Kernels? ICML, 2025.

35. J. Nie, et al.: KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardware, ICML, 2026.

36. B. Driscoll, et al.: Equivalence Checking of ML GPU Kernels (VOLTA), OOPSLA, 2026.

37. Y. Mei, et al.: Helix: Jointly optimizes model placement and request scheduling across heterogeneous GPUs and networks using a max-flow formulation, ASPLOS, 2025.

38. Y. Xian, et al.: Aegaeon: Uses token-level scheduling and autoscaling to efficiently share a GPU pool among multiple models with uneven demand, SOSP, 2025.

 

 2026/11/25 Session 7: Optimisations in ML Compiler

 

ML Compiler, SuperOptimisation, Hi-dimensional Parameter Space, Phase Ordering Problem, Reinforcement Learning.


1. Trofin, M. et al.: MLGO: a Machine Learning Guided Compiler Optimizations Framework, ArXiv, 2021.

3. Ma, L. et al.: Rammer: Enabling Holistic Deep Learning Compiler Optimizations with rTasks, OSDI, 2020.

4. Zheng, L. et al.: EinNet: Optimizing Tensor Programs with Derivation-Based Transformations, OSDI, 2023.

5. Nakandala, S. et al.: A Tensor Compiler for Unified Machine Learning Prediction Serving, OSDI, 2020.

6. Ding, Y. et al.: Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs, ASPLOS, 2023.

7. T. Chen et al.: Learning to Optimize Tensor Programs, NIPS, 2018.

8. T. Chen, T. Moreau, Z. Jiang, L. Zheng, S. Jiao, E. Yan, H. Shen, M. Cowan, L. Wang, Y. Hu, L. Ceze, C. Guestrin, and A. Krishnamurthy: TVM: An Automated End-to-End Optimizing Compiler for Deep Learning, OSDI, 2018. 

9. T. Chen, T. Moreau, Z. Jiang, L. Zheng, S. Jiao, E. Yan, H. Shen, M. Cowan, L. Wang, Y. Hu, L. Ceze, C. Guestrin, and A. Krishnamurthy: TVM: End-to-End Compilation Stack for Deep Learning, SysML, 2017.

11. Z. Jia, O. Padon, J. Thomas, T. Warszawski, M. Zaharia,  A. Aiken: TASO: Optimizing Deep Learning Computation with Automated Generation of Graph Substitutions: SOSP, 2019.

12. KH. Wang, J. Zhai, M. Gao, Z. Ma, S. Tang, L. Zheng, Y. Li, K. Rong, Y. Chen, and Z. Jia: PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections, ODSI, 2021.

13. A. Qiao, S. K. Choe, S. Subramanya, W. Neiswanger, Q. Ho, H. Zhang, G. R. Ganger, E. Xing: Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning, OSDI, 2021.

15. L. Zheng, et al.: TenSet: A Large-scale Program Performance Dataset for Learned Tensor Compilers, NeurIPS, 2021.  

17. R. Senanayake, et al.: A sparse iteration space transformation framework for sparse tensor algebra, OOPSLA, 2020.  

18. L. Zheng, et al.: Ansor : Generating High-Performance Tensor Programs for Deep Learning, OSDI, 2020.  

20. S. Zheng, et al.: FlexTensor: An Automatic Schedule Exploration and Optimization Framework for Tensor Computation on Heterogeneous System, ASPLOS, 2020.  

21. A. Paliwal et al.: REGAL: Transfer Learning For Fast Optimization of Computation Graphs, arxiv, 2019.  

22. F. Kjolstad, et al.: The tensor algebra compiler, OOPSLA, 2017.  

23. C. Cummins, et al.: Meta Large Language Model Compiler: Foundation Models of Compiler Optimization, Meta, 2024.  

24. J. Magalhães, et al.: C2taco: Lifting tensor code to taco, GPCE, 2023.  

25. C. Hvarfner, et al.: Vanilla Bayesian Optimization Performs Great in High Dimensions, PMLR, 2024.  

26. D. Eriksson, et al.: Scalable Global Optimization via Local Bayesian Optimization, NeurIPS, 2019.  

27. Y. Wu, et al.: Swift: Compiled Inference for Probabilistic Programming Languages, IJCAI, 2016.  

28. L. Wang, et al.: TileLang: Provides a tile-level programming model with compiler-assisted configuration inference for writing efficient fused GPU kernels, ICLR, 2026.

29. M. Wu, et al.: Mirage: Automatically discovers tensor optimizations across kernel, thread-block, and thread levels, combining algebraic and scheduling transformations, OSDI, 2025.

30. X. Cheng, et al.: MPK: Compiles multi-GPU inference into a persistent mega-kernel, enabling fine-grained scheduling and overlap between computation and communication, OSDI, 2026.

31. T. Zadouri, et al.: FlashAttention-4 : Optimizes attention for Blackwell GPUs through asynchronous tensor-core pipelines, tensor memory, and reductions in softmax and shared-memory overhead, arXiv, 2026.

32. Z. Ye, et al.: FlashInfer: Provides customizable attention kernels with efficient KV-cache access and load-balanced GPU scheduling for dynamic inference workloads, MLSys, 2025.

33. R. Sanovar, et al.: LeanAttention: Redesigns decode attention execution to improve GPU utilization and scale efficiently to long contexts, MLSys, 2025.

 

 Additional Reference Papers

 

See the additional papers in optimisation in Computer Systems for extended reading. Here!



 2025/12/03 Session 8: Presentation of Open Source Project Study

      
Presentation of Mini Project (work in progress, plan).

Wrap-up!              

                       

Coursework 1 (Reading Club)

The reading club will require you to read between 1 and 3 papers every week. You need to fill out simple review_log (MS word format, text format) prior to each session and email me by a day before - Tuesday (noon). The minimum requirement of review_log is one per session, but you can read as many as you want and fill the review_log for each paper you read.

At each session, 3 - 4 papers are selected under the session topic, and if you are assigned to present your review work, please prepare 15 minutes slides for presenting your review work. For each presentation, another student is assigned to prepare the discussion points (with one slide) and the discussion together with whole room will join the discussion (~8 minutes). Your presentation and discussion points materials should be emailed to me by the following day. You would present your review and discussion points approximately twice during the course. Note that take the following points when you create the slides.

  1. Presentation. The goal of this presentation is explaining the core concept of the paper from the viewpoint of the focused session topic to the other students. It is important to make it compact cutting off unnecessary details. 
    • Highlight the main design/architecture of the paper focused in the session
    • What is the significance from the existing works?
    • What is the novel idea?
  2. Discussion point.  
    • What is the limitation of the paper?
    • What is the alternative approaches?
    • What is required to complete the work?

 Coursework 2 (Reports)

The following two reports are required, both of which should address topics within the scope of R244.

  1. Survey report on sub-topic in data centric networking (~1800 - max 2000 words)
    • Pick up to ~5 papers as core papers in your survey scope
    • Read the above and expand your reading through related work
    • Comprehend your view and finish as your survey paper
    • See 'how to write a survey paper' in Assessment section.
  2. Project study and exploration of a prototype (max 2500 words)
    • What is the significance of the project in the research domain?
    • Compare with the similar and succeeding projects
    • Demonstrate the project by exploring its prototype
    • Please email your project selection (MS word format or text format <150 words) by 16:00 on November 20, 2026.
    • Project presentation on December 2, 2026.

The Survey report should be submitted by December 4, 2026 - 16:00 . The Project study report by January 20,  2027 - noon (Try to finish the mini project by the end of 2026!).

 Assessment

The final grade for the course will be provided as a letter grade or percentage and the assessment will consist of two parts:

  1. 35%: for reading club (Participation,  Presentation + tick of review_log and hands-on tutorial feedback - email me)
    • 20%: Presentation
    • 10%: Discussion point + Participation
    • 5%: Review log
  2. 65%: for the two reports (submit to moodle)
    • 25%: Survey report
    • 40%: Project study

Open Source Projects

See the candidates of Open Source Projects in data centric networking. The list is not exhausted. If you take anything other than the one in the list, please discuss with me. The purpose of this assignment is to understand the prototype of the proposed architecture, algorithms, and systems through running an actual prototype and present/explain to the other people how the prototype runs, any additional work you have done including your own applications and setup process of the prototype. This experience will give you better understanding of the project. These Open Source Projects come with a set of published papers and you should be able to examine your interests in the paper through running the prototype. Some projects are rather large and may require extensive environment and time; make sure you are able to complete this assignment.

How to Read/Review a Paper

The following papers aid how to read/review a paper.

Further supplement: see ‘how to read/review a paper’ section in Advanced Topics in Computer Systems by Steven Hand.

Presentations

Presentations should be about 15-20 minutes long, where you need to cover the following aspects.

  1. What are the background and the problem domain of the paper? What is the motivation of the presented work? What is the difference from the existing works?  What is the novel idea? How did the paper change/unchange the research in the research community?

  2. What is the significant contribution? How did the authors tackle the problem? Did the authors obtain expected result from their trial?

  3. How do you like the paper and why? What is the takeaway message to you (and to research community)? What is required to complete the work?

The following document aids in presenting a review.

How to write a survey paper

A survey paper provides the readers with an exposition of existing work that is comprehensive and organized. It must expose relevant details associated in the surveying area, but it is important to keep a consistent level of details and to avoid simply listing the different works. Thus a good survey paper should demonstrate a summary of recent research results in a novel way that integrates and adds understanding to work in the field. For example, you can take an approach by classifying the existing literature in your own way; develop a perspective on the area, and evaluate trends. Thus, after defining the scope of your survey, 1) classify and organize the trend, 2) critical evaluation of approaches (pros/cons), and 3) add your analysis or explanation (e.g. table, figure). Also adding reference and pointer to further in-depth information is important (summary from Rich Wolski’s note).

Papers for OS Principles (Distributed Storage and Deterministic Parallelism)

  • Following papers will help you to understand distributed storage and parallelism.
  • Systems Research and System Design
1.  B. Lampson: Hints for Computer Systems Design (Revised), ACM OSR 1983.

  • Distributed Storage
2. S. Ghemawat, H. Gobioff, and S. Leung: The Google File System, ACM SOSP 2003.
3. F. Chang et al: BigTable: A Distributed Storage System for Structured Data, USENIX OSDI 2006.
4. G. DeCandia et al:  Dynamo: Amazon's Highly Available Key-value Store, ACM SOSP 2007.

  • Deterministic Parallelism
5. J. Devietti et al: DMP: Deterministic Shared Memory Multiprocessing, ACM ASPLOS 2009.
6. A. Aviram, et al: Efficient System-Enforced Determistic Parallelism, USENIX OSDI 2010.
7. T. Liu et al: Dthreads: Efficient and Determistic Multithreading, ACM SOSP 2011.

Contact Email

Please email to eiko.yoneki@cl.cam.ac.uk for the question.