Geometric Deep Learning

Aims and Objectives

Most of the patterns we see in nature can be elegantly reasoned about using spatial symmetries—transformations that leave underlying objects unchanged. This observation has had direct implications for the development of modern deep learning architectures that are seemingly able to escape the “curse of dimensionality” and fit complex high-dimensional tasks from noisy real-world data. Beyond this, one of the most generic symmetries—the permutation—will prove remarkably powerful in building models that reason over graph structured-data, which is an excellent abstraction to reason about naturally-occurring, irregularly-structured data. Prominent examples include molecules (represented as graphs of atoms and bonds, with three-dimensional coordinates provided), social networks and transportation networks. Several already-impacted application areas include traffic forecasting, drug discovery, social network analysis and recommender systems. The module will provide the students the capability to analyse irregularly- and nontrivially-structured data in an effective way, and position geometric deep learning in a proper context with related fields. The main aim of the course is to enable students to make direct contributions to the field, thoroughly assimilate the key concepts in the area, and draw relevant connections to various other fields (such as NLP, Fourier Analysis and Probabilistic Graphical Models). We assume only a basic background in machine learning with deep neural networks.

Learning outcomes

• The framework of geometric deep learning, and its key building blocks: symmetries, representations, invariance and equivariance
• Fundamentals of processing data on graphs, as well as impactful application areas for graph representation learning
• Theoretical principles of graph machine learning: permutation invariance and equivariance
• The three "flavours" of spatial graph neural networks (GNNs) (convolutional, attentional, message passing) and their relative merits. The Transformer architecture as a special case.
• Attaching symmetries to graphs: CNNs on images, spheres and manifolds, Geometric Graphs and E(n)-equivariant GNNs
• Relevant connections of geometric deep learning to various other fields (such as NLP, Fourier Analysis and Probabilistic Graphical Models)

Lectures

The lectures will cover the following topics:

• Learning with invariances and symmetries: geometric deep learning. Foundations of group theory and representation theory.
• Why study data on graphs? Success stories: drug screening, travel time estimation, recommender systems. Fundamentals of graph data processing: network science, spectral clustering, node embeddings.
• Permutation invariance and equivariance on sets and graphs. Translation equivariance on grids. Rotation equivariance on spheres. Neural networks for point clouds: Deep Sets, PointNet.
• The three flavours of spatial GNNs: convolutional, attentional, message passing. Prominent examples: GCN, SGC, ChebyNets, MoNet, GAT, GATv2, IN, MPNN, GraphNets. Tradeoffs of using different GNN variants.
• Graph Rewiring: how to apply GNNs when there is no graph? Representative methodologies for graph rewiring: GDC, SDRF, EGP, DGCNN.
• Links to natural language processing---Transformers as a special case of attentional GNNs.
• Expressive power of Graph Neural Networks, the Weisfeiler-Lehman hierarchy, and beyond.
• Combining spatial symmetries with GNNs: E(n)-equivariant GNNs, Manifold/Mesh CNNs.
   

Practicals

The practical is designed to complement the knowledge learnt in lectures and teach students to derive additional important results and architectures not directly shown in lectures. The practical will be given as a series of individual exercises (each either code implementation or proof/derivation). Each of these exercises can be individually assessed based on a specified mark budget.

Possible practical topics include the study of higher-order GNNs and equivariant message passing.

Assessment

• (70%) Oral presentation accompanied with a 2,000-word report and source code, requiring a critical assessment of topical research paper(s), which needs to contain some implementational element (for example, this could be a reproducibility study or a incremental follow-up experiment).
• (30%) Practical work completion. Completing the exercises specified in the practical to a satisfactory standard. The practical assessor should be satisfied that the student derived their answers using insight gained from the course; coupled with original thought, not by simple copy-pasting of relevant related work. The students would submit code and a short report, which would then be marked in line with the predetermined mark budget for each practical item.

The students will learn how to run advanced architectures on GPU but no specific need for dedicated GPU resources. Practicals will be made possible to do on CPU; if required, students can use GPUs on publicly available free services (such as Colab) for their mini-project work.

References

The course will be based on the following literature:

• "Geometric Deep Learning: Grids, Graphs, Groups, Geodesics, and Gauges", by Michael Bronstein, Joan Bruna, Taco Cohen and Petar Veličković
• "Graph Representation Learning", by Will Hamilton
• "Deep Learning", by Ian Goodfellow, Yoshua Bengio and Aaron Courville.

Notes

This module will NOT be offered in 2026-27