# Petar Veličković

Hello, and welcome to one of my homes on the web (my personally hosted blog being a less formal one)!

I am currently completing my PhD in Machine Learning and Bioinformatics within the Artificial Intelligence Group of the Computer Laboratory, under supervision of Prof Pietro Liò, and I am affiliated with Trinity College. From January 2019, I am a full-time Research Scientist at DeepMind, working with Dr Raia Hadsell.

My research interests involve devising neural network architectures that operate on nontrivially structured data (such as graphs), and their applications in bioinformatics and medicine.

I hold an MA degree in Computer Science from the University of Cambridge, having completed the Computer Science Tripos in 2015. Between January 2016 and January 2019 I have also been employed as a Research Assistant in Computational Biology at the Department.

I am a proud recipient of the Computer Laboratory Wiseman Award for 2017, for my contribution to the work of the Department (through supervision of coursework and student projects, and developing and running practical lab sessions).

My Erdős number is 4, and can be derived in the following way: Paul Erdős → Nathan Linial → Ran El-Yaniv → Yoshua Bengio → Petar Veličković.

## Publications

**Veličković, P.** (2019) **The resurgence of structure in deep neural networks**. *Doctoral thesis*, doi:10.17863/CAM.39380

Deac, A., Huang, YH., **Veličković, P.**, Liò, P. and Tang, J. (2019) **Drug-Drug Adverse Effect Prediction with Graph Co-Attention**. *Workshop on Computational Biology at the 36 ^{th} International Conference on Machine Learning (ICML 2019)*. [arXiv preprint]

**Veličković, P.**, Fedus, W., Hamilton, WL., Liò, P., Bengio Y. and Hjelm, RD. (2019) **Deep Graph Infomax**. *The 7 ^{th} International Conference on Learning Representations (ICLR 2019)*. [arXiv preprint] [slides] [poster] [ZDNet coverage] [AITechnologies coverage]

This work has also been presented at the 2018 NeurIPS Relational Representational Learning (R2L) Workshop.

Opolka, FL.*, Solomon, A.*, Cangea, C., **Veličković, P.**, Liò, P. and Hjelm, RD. (2019) **Spatio-Temporal Deep Graph Infomax**. *Workshop on Representation Learning on Graphs and Manifolds (RLGM) at the 7 ^{th} International Conference on Learning Representations (ICLR 2019)*. [arXiv preprint]

Cangea, C.*, **Veličković, P.***, Jovanović, N., Kipf, T. and Liò, P. (2018) **Towards Sparse Hierarchical Graph Classifiers**. *Workshop on Relational Representation Learning (R2L) at the 32 ^{nd} Annual Conference on Neural Information Processing Systems (NeurIPS 2018)*. [arXiv preprint] [poster]

Deac, A., **Veličković, P.** and Sormanni, P. (2018) **Attentive cross-modal paratope prediction**. *Journal of Computational Biology*, doi:10.1089/cmb.2018.0175. [arXiv preprint] [poster]

This work has also been presented at the 2018 ICML/IJCAI Workshop on Computational Biology.

**Veličković, P.**, Cucurull, G., Casanova, A., Romero, A., Liò, P. and Bengio, Y. (2018) **Graph Attention Networks**. *The 6 ^{th} International Conference on Learning Representations (ICLR 2018)*. [arXiv preprint] [poster] [blog]

Cucurull, G., Wagstyl, K., Casanova, A., **Veličković, P.**, Jakobsen, E., Drozdzal, M., Romero, A., Evans, A. and Bengio, Y. (2018) **Convolutional neural networks for mesh-based parcellation of the cerebral cortex**. *The 1 ^{st} Conference on Medical Imaging with Deep Learning (MIDL 2018)*. [OpenReview preprint]

This work has also been presented at the

*2017 NIPS BigNeuro Workshop*.

**Veličković, P.**, Karazija, L., Lane, ND., Bhattacharya, S., Liberis, E., Liò, P., Chieh, A., Bellahsen, O. and Vegreville, M. (2018) **Cross-modal recurrent models for weight objective prediction from multimodal time-series data**. *The 12 ^{th} EAI International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth 2018)*, doi:10.1145/3240925.3240937. [arXiv preprint] [poster]

This work has also been presented at the

*2017 NIPS Workshop on Machine Learning for Health*, the

*2017 NIPS Time Series Workshop*and

*ObesityWeek 2017*.

Liberis, E., **Veličković, P.**, Sormanni, P., Vendruscolo, M. and Liò, P. (2018) **Parapred: Antibody Paratope Prediction using Convolutional and Recurrent Neural Networks**. *Bioinformatics*, doi:10.1093/bioinformatics/bty305. [bioRxiv preprint]

Karazija, L., **Veličković, P.** and Liò, P. (2018) **Automatic inference of cross-modal connection topologies for X-CNNs**. *The 15 ^{th} International Symposium on Neural Networks (ISNN 2018)*, doi:10.1007/978-3-319-92537-0_7. [arXiv preprint]

Bica, I., **Veličković, P.**, Xiao, H. and Liò, P. (2018) **Multi-omics data integration using cross-modal neural networks**. *The 26 ^{th} European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2018)*.

Mathur, A., Zhang, T., Bhattacharya, S., **Veličković, P.**, Joffe, L., Lane, ND., Kawsar, F. and Liò, P. (2018) **Using deep data augmentation training to address software and hardware heterogeneities in wearable and smartphone sensing devices**. *The 17 ^{th} ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN 2018)*, doi:10.1109/IPSN.2018.00048.

Peychev, M., **Veličković, P.** and Liò, P. (2017) **Quantifying the Effects of Enforcing Disentanglement on Variational Autoencoders**. *Workshop on Learning Disentangled Representations at the 31 ^{st} Annual Conference on Neural Information Processing Systems (NIPS 2017)*. [arXiv preprint] [poster]

Cangea, C., **Veličković, P.** and Liò, P. (2017) **XFlow: 1D-2D Cross-modal Deep Neural Networks for Audiovisual Classification**. *Workshop on Computational Models for Crossmodal Learning (CMCML) at The 7 ^{th} Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (IEEE ICDL-EPIROB 2017)*. [arXiv preprint] [poster]

This work has also been presented at the

*ARM Research Summit 2017*.

**Veličković, P.**, Lane, ND., Bhattacharya, S., Chieh, A., Bellahsen, O. and Vegreville, M. (2017) **Scaling health analytics to millions without compromising privacy using deep distributed behaviour models**. *The 11 ^{th} EAI International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth 2017)*, doi:10.1145/3154862.3154873.

**Veličković, P.**, Wang, D., Lane, ND. and Liò, P. (2016) **X-CNN: Cross-modal convolutional neural networks for sparse datasets**. *The 7 ^{th} IEEE Symposium Series on Computational Intelligence (IEEE SSCI 2016)*, doi:10.1109/SSCI.2016.7849978 [arXiv preprint] [slides] [poster]

This work has also been presented at the

*ARM Research Summit 2016*, the

*ARM-Cambridge Research Showcase*and the

*Fourth Edinburgh Deep Learning Workshop*.

**Veličković, P.**, Ivašković, A., Lau, S. and Stanojević, M. (2016) **Viral: Real-world competing process simulations on multiplex networks**. *The 1 ^{st} Belgrade Bioinformatics Conference (BelBi 2016)*.

**Veličković, P.** and Liò, P. (2016) **muxstep: an open-source C++ multiplex HMM library for making inferences on multiple data types**. *Bioinformatics*, doi:10.1093/bioinformatics/btw196

**Veličković, P.** and Liò, P. (2015) **Molecular multiplex network inference using Gaussian mixture hidden Markov models**. *Journal of Complex Networks*, doi:10.1093/comnet/cnv029

## Software

**facejack**: a Python application highlighting the perils of deploying neural networks as components of secure authentication systems, leveraging adversarial learning as an attack vector. *Third prize at Hack Cambridge Recurse.*

**Viral**: Java-based client-server application that facilitates real-life simulations of epidemics (and other spreading processes) in modern society, using a multilayer network back-end. *One of the seven finalist projects at Hack Cambridge*.

**muxstep**: an open-sourced generalised C++ library implementation of the Multiplex (GM)HMM model proposed in previous research, designed to be immediately useful out-of-the-box as well as easily extendable for specific user needs.

## Lecturing

Thus far, I have held five formal lectures to undergraduate and postgraduate students. I enjoy lecturing, and would definitely be interested in further opportunities within this scope.

**Be nice to your neurons: Initialisation, Normalisation, Optimisation and Regularisation**. Introduction to Deep Learning (COMPGI23), *University College London*, 14 November 2017.

**Unsupervised methods: Diving deep into autoencoders**. Introduction to Deep Learning (COMPGI23), *University College London*, 10 October 2017.

**A trip down long short-term memory lane**. Research Students Lecture Series, 21 February 2017.

**Exactly solving TSP using the Simplex algorithm** (with Thomas Sauerwald). Advanced Algorithms, 13 May 2016, 17 May 2017.

Furthermore, I have been in charge of designing the practical session on neural networks for the MPhil course on Machine Learning and Algorithms for Data Mining (L42).

## Talks

**Adversarial learning meets graphs (and why should you care?)**. University of Oxford Cyber Physical Systems Seminars, 8 January 2019. *Department of Computer Science, University of Oxford*
Also presented at the MILA Adversarial Learning Group Meeting, 25 September 2018. *Montréal Institute for Learning Algorithms*

**Relational Learning**. MILA Graph Representation Learning Group Meeting, 22 August 2018. *Montréal Institute for Learning Algorithms*

**Keeping our graphs attentive**. MILA Tea Talk Series, 3 August 2018. *Montréal Institute for Learning Algorithms*

Also presented (occasionally under the name *"Towards practical convolutional neural networks on graphs"*) at the following venues:

- Microsoft Research Montréal Seminar, 28 September 2018.
*Microsoft Research Montréal* - Cambridge Computational Biology Institute Annual Symposium, 16 May 2018.
*Centre for Mathematical Sciences, University of Cambridge* - Randomized Algorithms Reading Group Meeting, 7 May 2018.
*Computer Laboratory, University of Cambridge* - Amsterdam Machine Learning Lab Seminar, 10 April 2018.
*Universiteit van Amsterdam* - Trinity Mathematical Society Symposium, 26 February 2018.
*Winstanley Lecture Theatre, Trinity College, Cambridge*

**Neural Relational Inference for Interacting Systems**. MILA Graph Representation Group Meeting, 25 July 2018. *Montréal Institute for Learning Algorithms*

**Generative Adversarial Networks**. Pint of Science Cambridge: I think, therefore AI, 15 May 2018. *The Panton Arms, Cambridge*

**Overview of neural network architectures for graph-structured data analysis**. UCL AI Journal Club, 19 February 2018. *University College London*

Also presented at the Computational Biology Group Meeting, 21 November 2017. *Department of Computer Science and Technology, University of Cambridge*

**Demystifying deep learning**. London Data Science Summit, 20 October 2017. *Royal Statistical Society*

Also presented at the DIAL Seminar Series, 17 May 2017. *Institute for Manufacturing, University of Cambridge*

**How to get a neural network to do what you want: Loss function engineering**. Trinity College Engineering Society Talk Series, 3 March 2017. *Winstanley Lecture Theatre, Trinity College, Cambridge*

**How to Win a Hackathon**. Hackers at Cambridge Events, 2 March 2017. *Computer Laboratory, University of Cambridge*

**Reinforcement learning fundamentals**. Nokia Tech Talk Series, 11 October 2016. *Nokia Technologies Cambridge*

**Exploring large-scale reinforcement learning tasks? Exploit deep learning.**. Cambridge Coding Academy Evening Tech Talk Series, 20 September 2016. *Google Campus London*

**AlphaGo: Superhuman artificial intelligence in exponentially growing spaces**. Cambridge Coding Academy Evening Tech Talk Series, 25 August 2016. *THECUBE London*

**How to make a (GameBoy) emulator?**. Cambridge Coding Academy Summer School, 13 July 2016. *Computer Laboratory, University of Cambridge*

**Deep reinforcement learning: DQN and extensions**. NVIDIA Deep Learning Day, 30 June 2016. *New Museums Site, University of Cambridge*

**Competitive programming**. Computational Biology Group Meeting, 5 May 2016. *Computer Laboratory, University of Cambridge*

**Viral: Realne simulacije konkurentnih procesa nad multipleks mrežama** (with Andrej Ivašković). Bioinformatics Seminar, 6 April 2016. *Faculty of Mathematics, University of Belgrade*

**Multiplex network inference (using hidden Markov models)**. Computational Biology Group Meeting, 11 February 2016. *Computer Laboratory, University of Cambridge*

## Supervisions

I currently supervise, have supervised, or will be supervising, the project work of the following students (with completed dissertations, where possible, available for download by anyone with a Raven account):

- Benjamin Day (2016)
**Combining Machine Learning & Multilayer Network Theory**.*Natural Sciences Tripos Part III Computational Project (co-supervised*). Project mark 91/100* - Ioana Bica (2017)
**Comparative Analysis of Neural Network Architectures for Epigenetics Inference**.*Computer Science Tripos Part II Project Dissertation. Project mark 73/100* - Momchil Peychev (2017)
**Experimental Study on the Properties of Disentangled Autoencoders**.*Computer Science Tripos Part II Project Dissertation. Project mark 82/100* - Tudor Petru Ţiplea (2017)
**An end-to-end control policy system for the GameBoy platform**.*Computer Science Tripos Part II Project Dissertation. Project mark 87/100***(Highly commended dissertation)** - Edgaras Liberis (2017)
**Antibody Binding Residue Prediction**.*Computer Science Tripos Part III Project Dissertation (co-supervised*). Project mark 86/100* - Laurynas Karazija (2017)
**Automatic inference of cross-modal connection topologies for X-CNNs**.*Computer Science Tripos Part III Project Dissertation (co-supervised*). Project mark 89/100***(Dr John Maheswaran Prize for a Highly Commended Project Report)** - Cătălina Cangea (2017)
**1D-2D Cross-modality for deep audiovisual classification**.*MPhil in Advanced Computer Science Project Dissertation (co-supervised*). Project mark 84/100* - Andreea-Ioana Deac (2018)
**Neural architectures for end-to-end paratope prediction**.*Computer Science Tripos Part II Project Dissertation. Project mark 79/100* - Miloš Stanojević (2018)
**Neural network hyperparameter optimisation on small validation sets**.*Computer Science Tripos Part III Project Dissertation (co-supervised*). Project mark 79/100* - Nathaniel McAleese-Park (2018)
**Cross-modal learning for the joint understanding of 2D and 3D data**.*Computer Science Tripos Part III Project Dissertation (co-supervised*). Project mark 81/100* - Aliyah Bond (2019)
**Intelligent navigation of text adventure games**.*Computer Science Tripos Part II Project Dissertation.* - Carlos Purves (2019).
**The PlayStation reinforcement learning environment**.*Computer Science Tripos Part II Project Dissertation.* - Aaron Solomon (2019).
**Dynamic temporal analysis for graph structured data**.*MPhil in Advanced Computer Science Project Dissertation (co-supervised*).* - Felix Opolka (2019).
**Representation learning for spatio-temporal graphs**.*MPhil in Advanced Computer Science Project Dissertation (co-supervised*).*

*for "co-supervised" projects, my name has not been listed as one of the official supervisor(s) (primarily due to constraints on eligibility for supervisors of Master's level projects). I have, however, made a significant contribution to the supervision of these projects, and will be listed as a co-author on any relevant subsequent publications.

I have supervised the following courses:

- Advanced Algorithms (Easter 2016, Easter 2017)
- Algorithms (Lent 2017, Lent 2018)
- Artificial Intelligence (Easter 2018)
- Artificial Intelligence II (Lent 2016)
- Bioinformatics (Michaelmas 2016, Michaelmas 2017)
- Compiler Construction (Lent 2016, Lent 2017)
- Complexity Theory (Easter 2017, Easter 2018)
- Computation Theory (Lent 2016, Lent 2017)
- Concepts in Programming Languages (Easter 2016)
- Concurrent and Distributed Systems (Michaelmas 2016, Lent 2017)
- Databases (Lent 2016, Lent 2017)
- Discrete Mathematics (Michaelmas 2017, Lent 2018)
- Foundations of Data Science (Michaelmas 2017)
- Further Java (Michaelmas 2016)
- Logic and Proof (Lent 2017)
- Machine Learning and Bayesian Inference (Lent 2017, Lent 2018)
- Mathematical Methods for Computer Science (Michaelmas 2016)
- Numerical Methods (Easter 2017)
- Programming in C and C++ (Michaelmas 2016)
- Quantum Computing (Michaelmas 2017)
- Security I (Easter 2016)
- Semantics of Programming Languages (Michaelmas 2016)

As a product of preparing these supervisions, I have also created a set of example sheets and auxiliary materials. I'll leave them here, so that they can be found more easily.

- Advanced Algorithms - k-best 0-1 Knapsack Problem
- Algorithms - Example Pre-Sheet
- Algorithms - Example Sheet 1
- Algorithms - Example Sheet 2
- Algorithms - Example Sheet 3
- Algorithms - Example Sheet 4
- Algorithms - Example Sheet 5
- Algorithms - Example Sheet 6
- Bioinformatics - Example Sheet 1
- Bioinformatics - Example Sheet 2
- Bioinformatics - Example Sheet 3
- Bioinformatics - Expectation Maximisation Notes
- Compiler Construction - Example Sheet 1
- Compiler Construction - Example Sheet 2
- Compiler Construction - Example Sheet 3
- Compiler Construction - Example Sheet 3++
- Compiler Construction (+ Databases) - Example Sheet 4
- Compiler Construction (+ Databases) - Example Sheet 5
- Security I - Example Sheet 1
- Security I - Example Sheet 2

## Work experience

I have attained substantial work experience spanning a wide variety of Computer Science fields, through six summer internships and other placements during my undergraduate and PhD degrees:

- 2018: I have returned to the Montréal Institute for Learning Algorithms as a visiting researcher, working with Dr Devon Hjelm and Prof Yoshua Bengio. I worked on information-theoretic approaches to unsupervised representation learning on graph-structured data, recovering the Deep Graph Infomax (DGI) algorithm as a result.
- 2017: I have been a visiting researcher at the Montréal Institute for Learning Algorithms (MILA) over the summer, working with Dr Adriana Romero and Prof Yoshua Bengio. I focussed on devising novel neural network architectures for graph-structured data analysis, ultimately developing Graph Attention Networks (GATs), and contributing to applications in brain mesh segmentation.
- 2016: I have worked as a research associate at Nokia Bell Labs, as part of a continued (and ongoing) collaboration with Dr Nic Lane. The result of this work are four (so far!) papers relevant to the scope of my PhD.
- 2015: I have worked as a software developer intern at Jane Street. This allowed me to experience first-hand one of the major real-world uses of functional programming, and particularly how it's used for efficient quantitative trading. I have worked directly on trading-specific critical systems, as well as on infrastructure and standard library development (contributing directly to Jane Street's Core standard library), primarily in OCaml. My work was highly commended.
- 2014: I have worked on the Altera-sponsored summer placement at the Computer Laboratory's Computer Architecture Group, under the supervision of Prof Simon Moore. I have worked in parallel on developing debugging and tracing utilities as part of the ongoing CHERI research project, as well as developing potential future teaching resources. The projects have been successfully executed.
- 2013: I have interned as a software engineer at Microsoft Development Center Serbia, where I was part of a four-man team in charge of creating a prototype for a brand new stress testing tool; it was my job to handle the entire implementation and present the results achieved to the test leads in the US; the project was extremely successful.

## Interests

I am highly active as a machine learning educator. To this end, I collaborated heavily with Cambridge Spark, a Cambridge-based startup for continual and professional development. I contributed to their work in the following ways, ordered (roughly) chronologically:

- Writing an introductory tutorial series on machine learning with deep neural networks using Keras, titled "Deep learning for complete beginners". The series has reached a reasonably high level of popularity online, being featured in at least one "top-ten" list. Unfortunately, the links to the series are now broken. :(
- Delivering two evening tech talks in London, focussing on current machine learning trends, and accessible to the wider public (both full house events):
- AlphaGo: Superhuman artificial intelligence in exponentially growing spaces
- Exploring large-scale reinforcement learning tasks? Expoit deep learning.

- Initially tutoring, and then also helping the continual development of the Machine Learning Techniques using Python bootcamp. I have tutored variants of it in London, Cambridge and Glasgow, to various bootcamp cohorts as well as several industry clients.
- Leading the development of the "Neural Networks and Deep Learning" module of the Applied Data Science part-time course, to be offered for the first time starting October 2017.
- Giving a talk on demystifying deep learning, as well as leading a practical workshop on neural networks and deep learning, at the London Data Science Summit 2017 organised by Cambridge Spark and the Royal Statistical Society.

Along with Andrej Ivašković and Miloš Stanojević, in 2015 I have kickstarted the mgcsweek seminar at the Mathematical Grammar School in Belgrade, and was actively engaged with its organisation until April 2018. This seminar aims to give insights into a variety of Computer Science topics, to ~20 of the most gifted high school students, for a week during each school year. Through this seminar I was given the excellent opportunity to give lectures on several topics in Computer Science I find particularly interesting:

- Bioinformatics algorithms: analysis of DNA sequences
- How to make a (GameBoy) emulator?
- Computer security: authentication protocols and passwords
- Great software disasters
- Linear programming and the Simplex algorithm
- IQ sampling and FM demodulation using MATLAB
- Markov chains, PageRank and clustering
- Multiplex network inference
- AlphaGo: superhuman artificial intelligence in exponentially growing spaces
- Data compression on DNA molecules
- Anatomy of an electronic trading system
- Neural networks
- Deep learning workshop in Keras
- Security in IoT and AI
- Reinforcement Learning

Thus far the seminar has been very successful and has expanded far beyond our original conceptions, with the potential of spreading across the specialised high schools in Serbia. It was covered by the Serbian national television (RTS) twice: (mgcsweek1), (mgcsweek2), as well as by the Serbian Science Television (mgcsweek3).

I used to be an extremely avid competitive programmer (and still compete on Codeforces occasionally). Achievements I'm happy about on that front include:

- Silver medal at the ACM-ICPC Northwestern Europe Regional Contest 2016 in Bath, United Kingdom;
- Bronze medal at the ACM-ICPC Northwestern Europe Regional Contest 2014 in Linköping, Sweden;
- Ranked 5th at the Final Round of the Google Hash Code 2017 contest in Paris, France;
- Four-time finalist at the Bubble Cup contest, organised by Microsoft Development Center Serbia (in 2012, 2013, 2014 and 2016);
- Ranked 311st at the Facebook Hacker Cup 2016 Round 2 (sufficient to win a Facebook t-shirt);
- Ranked 2nd overall at Codeforces Round #169 (Div. 2);
- Third prize at the Serbian Olympiad in Informatics in 2012.

Between 2014 and 2018, I was a member of the Serbian Committee for Competitions in Informatics, which is responsible for organising contests on all levels from regional all the way to the Serbian Olympiad in Informatics, determining Serbia's team for the International Olympiad in Informatics.

I was also proud to be serving as the coach for all ACM-ICPC teams from Cambridge for 2017. One of my teams, Me[N]ta∭ca, has won the ACM-ICPC Northwestern Europe Regional Contest 2017 (in Bath, United Kingdom), and ranked 31st at the 2018 ACM-ICPC World Finals (in Beijing, China)!

Aside from this, I also enjoy hackathons. I've attended seven of them so far, with Hack Cambridge being the most successful one by far (having made it into the top seven projects for two years straight, winning a 3rd prize in 2017, as well as spearheading the project that ended up becoming Viral).

## Contact

**Address:**

Petar Veličković

Office FE14

Artificial Intelligence Group

Computer Laboratory

University of Cambridge

15 JJ Thomson Avenue

Cambridge CB3 0FD

United Kingdom

**Phone:** +44 (0)1223 763544

**Email:** petar [dot] velickovic [at] cst [dot] cam [dot] ac [dot] uk