Research News...

You might be here looking for Pico, our lightweight research framework for designing and pretraining SLMs. See Richard present the video tutorial here (video production Zeb Goriely).

University of Cambridge

I am Deputy Head of Department and Professor of Language and Machine Learning in the Department of Computer Science and Technology

For my publications see scholar.

Research Interests

We are currently developing Pico, our research framework for designing and pretraining performant Small (and Baby) Language Models. Smaller language models are essential when conducting research with small, confidential and bespoke datasets. Using Pico, we are conducting theoretical work that investigates the performance, interpretability and biases associated with Language Models, and we explore cognitively motivated model design. This is important work for equalising opportunities in AI.

More generally, I am a computational linguist interested in fundamental questions about language: why have languages evolved to be what they are today? how do we learn languages? how can we learn second languages most efficiently? how should we organise language-related information for particular tasks?

Together with my research team, we develop Natural Language Processing and Machine Learning techniques to research language cognition (computational psycholinguistics). We build cognitively-motivated language applications and we research explanatory AI (that is, providing stake-holder appropriate explanations of the decisions AI has made).

Other work has focused on building Natural Language Processing tools that work with non-canonical forms of natural language (spoken language, learners, aphasics, social media language) and also with low resource languages (endangered languages, dialects). We are interested in both the automatic machine processing of language and the cognitive processes underlying that language; we have built models of artificial learners for first and second language learning.

We are also interested in the automatic extraction of units of information from language excerpts (whether written or spoken) and how to represent relationships between these units.

The team of wonderful research students and researchers that I('ve) line manage(d): Oistein Andersen, Luca Benedetto, Christian Bentz, Chris Bryant, Andrew Caines, Chris Davis, Richard Diehl Martinez, Mark Elliott, Mariano Felice, Gabrielle Gadeau, Calbert Graham, Felix Hill, Tim Luka Horstmann, Diana Galvan Sosa, Zeb Goriely, Philip Moore, Russell Moore, Marek Rei, Suchir Salhan, David Strohmaier, Shiva Taslimipoor, Gladys Tyen, Helen Yannakoudakis, Zheng Yuan, Ahmed Zaidi, ... (apologies if I missed someone -- let me know)

University Roles

I have several roles in the University: Professor of Language and Machine Learning in the Department of Computer Science and Technology; co-Director of the Cambridge Language Sciences Interdisciplinary Research Centre; Principal Investigator of the Cambridge Institute for Automated Language Teaching and Assessment (ALTA), which is an Artificial Intelligence institute that uses techniques from Machine Learning and Natural Language Processing to improve the experience of learning and assessment online. I was previously Lead Scientific Advisor to Cambridge University Press and Assessment.

Industry -- University Spinouts

I am part of iLexIR, a University spinout that provides natural language processing solutions---specializing in text analytics, mining, classification and search applications.

I am Chief Scientist for RegGenome, a University spinout formed by colleagues from the Cambridge Judge Business School.

Information for Prospective Students

Students interested in working within my fields of research may want to apply to the MPhil in Advanced Computer Science (ACS). I very rarely accept PhD students who haven't first completed the ACS (or equivalent). If you wish to apply for a PhD, please look for application information on the department web pages. Please do not email me a CV attachment. Summer internship positions are sometimes possible for Cambridge University students but not more widely. To let me know you have read this page before mailing me, please include the word Jabberwock in the subject header.