[Research] [Blog] [Videos] [Politics] [My Book] [Music] [Seminars] [Contact Details]
Chat Control or Child Protection analyses the arguments used by GCHQ that we should circumvent the end-to-end crypto in messenger apps "to protect children" and shows that they are not consistent with the evidence. Their arguments are used to support the government's Online Safety Bill which would give Ofcom a lot of powers it shouldn't have, including the power to mandate snooping software in your phone. If Parliament really cares about child safety and terrorism, what should it do instead? (blog evidence video)
One Protocol to Rule Them All? On Securing Interoperable Messaging analyses the EU DMA mandate for messaging systems interoperability. This will vastly increase the attack surface at every level in the stack – from the cryptography up through usability to commercial incentives and the opportunities for government interference. It will be complexity on steroids (blog Register Schneier).
Talking Trojan describes what we
learned from trying to get industry to fix the Trojan Source vulnerability,
which broke almost all computer languages, and the related Bad Characters
vulnerability, which broke almost all NLP models. What parts of the disclosure
ecosystem work, and which are broken? (blog)
ExtremeBB is a database we have
collected of more than 50m posts to underground extremist forums, and which we
make available to social scientists studying violent online political extremism,
misogyny, radicalisation and hate speech. This exercise has taught us about the
remarkably strong correlation between misogyny and terrorist violence, whether
from Islamists or the far right (blog).
CoverDrop: Securing Initial Contact for Whistleblowers is a prototype we built of a better way for news organisations to support whistleblowers. The most dangerous step for a whistleblower can be making initial contact; by building end-to-end encrypted chat into its news app, a newspaper can make this much more unobtrusive (blog).
Trojan Source: Invisible Vulnerabilities shows how to make software vulnerabilities invisible to human reviewers, by using Unicode tricks that are ignored by compilers but which cause editors to display an altered version of the software under review. This affected most modern computer languages when we disclosed it, and still affects some of them (blog website code Rust blog Schneier Krebs video).
Bad Characters: Imperceptible NLP Attacks shows how the systems used for common natural-language processing tasks such as machine translation and toxic content filtering can be broken easily by inputs with adversarial coding. This can enable bad actors to hide in plain sight (website code).
Bugs in our Pockets: The Risks of Client-Side Scanning is a thorough technical study of the risks involved in mandatory scanning of people's phones and other devices for illegal materials, as proposed in various forms by the US and UK governments, the EU and Apple (blog Guardian New York Times The Register FT Forbes Lawfare NetzPolitik El Pais Observer podcast)
Silicon Den: Cybercrime is Entrepreneurship analyses underground criminal enterprises as tech startups; their main impediment compared with regular tech businesses may be lack of access to finance (blog)
Markpainting: Adversarial Machine Learning meets Inpainting shows how to defeat inpainters – machine-learning tools that make it easy to edit or even forge images. Adversarial machine-learning tricks can be used to make images tamper-evident, or to add copyright marks that are extremely difficult for inpainters to remove (blog).
Manipulating SGD with Data Ordering Attacks shows that it's easy to poison or backdoor a machine-learning system without changing the training data; you only have to manipulate the order in which the training samples are presented (blog Schneier Doctorow).
I joined Edinburgh University as Professor of Security Engineering in February 2021. I'm now 20% at Edinburgh and 80% at Cambridge. I'm now teaching a course in Security Engineering at Edinburgh to masters students and fourth-year undergrads, and the lecture videos are now all online (as are the lecture videos and notes for my first-year undergrad course on Software and Security Engineering at Cambridge).
My main project in 2019 and 2020 was writing a third edition of my Security Engineering textbook.
At Cambridge University Computer Laboratory, my research students are Jenny Blessing,
Nicholas Boucher and David Khachaturov; my RAs
are Richard Clayton, Hridoy Dutta and Sergei Skorobogatov.
I have also been working with Tim
Clifford and Robert Brady.
My former RAs are Lydia Wilson, Franck Courbon, Anh Vu, Maria Bada, Yi Ting Chua, Ben Collier, Helen Oliver, Ildiko Pete, Daniel Thomas, Alice Hutchings, Sergio Pastrana, David Modic, Sven Übelacker, Julia Powles, Ramsey Faragher, Sophie van der Zee, Mike Bond, Vashek Matyas, Steven Murdoch, Andrei Serjantov and
Alex Vetterl. My former
students Jong-Hyeon Lee, Frank Stajano, Fabien Petitcolas, Harry
Manifavas, Markus Kuhn,
Ulrich Lang, Jeff Yan, Susan Pancho-Festin,
Mike Bond, George Danezis, Sergei Skorobogatov, Hyun-Jin Choi, Richard Clayton, Jolyon Clulow, Hao Feng, Andy Ozment, Tyler Moore, Shishir Nagaraja, Robert Watson, Hyoungshick Kim, Shailendra Fuloria, Joe Bonneau, Wei-Ming Khoo, Rubin Xu, Laurent Simon, Kumar Sharad, Shehar Bano, Dongting Yu, Khaled Baqer, Alex Vetterl, Mansoor Ahmed and Ilia Shumailov
have earned PhDs.
I'm teaching three Cambridge courses in 2022-23: the undergraduate course in Software
and Security Engineering and graduate courses in Computer
Security and Cybercrime. I also organise our security
seminars and help to run the Cambridge Cybercrime Centre.
My research topics include:
Machine learning and signal processing
The detection and manipulation of hidden information has many applications, and the field is being refreshed by the recent revolution in neural networks.
Talking Trojan describes what we
learned from trying to get industry to fix the Trojan Source vulnerability,
which broke almost all computer languages, and the related Bad Characters
vulnerability, which broke almost all NLP models. What parts of the disclosure
ecosystem work, and which are broken? (blog)
- Trojan Source: Invisible Vulnerabilities shows how adversarial coding can make source code look different to a compiler and to a human reviewer. This enables supply-chain attacks to hide in plain sight (website blog).
- Bad Characters: Imperceptible NLP Attacks shows how the systems used for common natural-language processing tasks such as machine translation and toxic content filtering can be broken easily by inputs with adversarial coding. This can enable bad actors to hide in plain sight (website code).
- Markpainting: Adversarial Machine Learning meets Inpainting shows how to defeat inpainters – machine-learning tools that make it easy to edit or even forge images. Adversarial machine-learning tricks can be used to make images tamper-evident, or to add copyright marks that are extremely difficult for inpainters to remove (blog).
- Situational Awareness and Machine Learning – Robots, Manners and Stress argues that manners are a new frontier for research in robotics and machine learning. ML models find it really hard to interact with multiple humans, for example when an autonomous vehicle is trying to turn across traffic; this is related to situational awareness (blog).
- Data Ordering Attacks enable you to poison or backdoor a machine-learning system without changing the training data; you only have to manipulate the order in which the training samples are presented. For example, you can train a credit-scoring algorithm to be sexist by starting its training with ten rich men and ten poor women; but it's very much more general than that (blog).
- Sponge Examples: Energy-Latency Attacks on Neural Networks describes how to find inputs to neural networks that make them take a lot of time, or burn a lot of energy. They can be used to distract or to jam machine learning systems in a wide range of applications (blog press Schneier).
- Blackbox Attacks on Reinforcement Learning Agents Using Approximated Temporal Information demonstrates delayed-action attacks on reinforcement learning agents; some might be used as time bombs.
- Nudge Attacks on Point-Cloud DNNs disturb a small number of input points to a DNN to change how it classifies a 3-d object, and may therefore cause an autonomous vehicle or other robot to misunderstand its environment. We show two ways to generate them.
- The Taboo Trap is a mechanism we invented to block adversarial machine learning attacks on energy-constrained devices. An older version of paper was the subject of my invited talk at AISEC 2019. It emerged from earlier work on neural network compression, which appeared at SysML.
- The Taboo Trap work also led to further papers on transferability and adversarial reinforcement learning.
- Hey Alexa what did I just type? Decoding smartphone sounds with a voice assistant shows that if you type a password or PIN on a mobile phone within half a metre of a smart speaker with a good directional microphone array, the taps can give a lot of information about what you typed (New Scientist, Bruce Schneier, John Naughton, Daily Mail).
- BatNet: Data transmission between smartphones over ultrasound shows how to build a censorship-resistant mesh network using ultrasonic signals between smartphones. We also tested this as a covid contact-tracing technique; it turned out to be just as flaky as Bluetooth.
- Hearing your touch describes a new way to hack phones. A phone screen, like a drum, makes slightly different sounds depending on where you tap it, and given two microphones you can locate the tap too. So a hostile app can recover PIN codes and short words given a few measurements (blog Schneier).
Interrupt Me While I Type: Inferring Text Entered Through Gesture Typing on
Android Keyboards demonstrates a new side channel that can enable one
Android app to steal another app's input
- In PIN Skimmer:
Inferring PINs Through The Camera and Microphone we show that software on
your smartphone can work out what PIN you enter on your phone by watching your
face through the camera and listening for the clicks as you type (blog BBC CNN).
- Soft Tempest: Hidden Data Transmission Using Electromagnetic Emanations showed that the software on a computer can control its stray electromagnetic emanations. This can be used for both attack and defence. There's also a followup paper on the costs and benefits of Soft Tempest.
- Hollywood once hoped that copyright-marking systems would help control the copying of videos, music and computer games. This became high drama when a paper that showed how to break the DVD/SDMI copyright marking scheme was pulled by its authors from the Information Hiding 2001 workshop, following legal threats from Hollywood. In fact, the basic scheme – echo hiding – was among a number that we broke in 1997: see Attacks on Copyright Marking Systems. We also wrote Information Hiding – A Survey, which is a good place to start. There is much more on the web page of my former student Fabien Petitcolas.
- Another novel application of information hiding is
File System. It will give you any file whose name and password you know,
but if you do not know the correct password, you cannot even tell that a file
of that name exists in the system! Its main function is to protect users
against coercion. Two of our students implemented it: a paper
is here, while the
code is here. This
functionality has since appeared in a number of crypto products.
- The threat in the 1990s by some governments to ban cryptography led to a
surge of interest in steganography – the art of hiding messages in other
messages – and then the surge of paranoia post-9/11 stoked interest in
looking for them, with nonsense
like this boosting many a bureaucrat's budget. Our paper On The Limits
of Steganography explored what can and can't be done (here's an earlier version).
- The Newton
Channel settles a conjecture of Simmons by exhibiting a high bandwidth
subliminal channel in the ElGamal signature scheme.
Sustainability of security
- Making security sustainable is the new grand challenge for computer science: designing software so that durable goods such as cars can last longer (video of talk at 36C3 blog).
- Standardisation and Certification in the Internet of Things is an analysis of what happens to safety regulation once we get software everywhere. It informed EU directive 2019/771 which requires firms selling goods with digital components to maintain the software for at least two years, or for the reasonable expectation of the customer if longer. This will probably mean ten years for cars and white goods (blog).
- Privacy for Tigers describes work we did to stop wildlife aggregation sites being exploited by poachers.
- Bitcoin Redux examines what’s gone wrong in the world of cryptocurrencies, whose mining wastes colossal amounts of energy; financial regulators bear some of the blame for failing to enforce existing laws that would have prevented some of the worst abuses (blog). It follows on from Making Bitcoin Legal, where we presented a better way of tracing stolen bitcoin (blog video).
- What you get is what you C describes a compiler plugin we wrote to make it easier to maintain crypto code by expressing programmer intent.
- DigiTally is a prototype payment system we built to extend mobile phone payments to areas of less developed countries with no phone service.
- The UK smart meter project looks set to waste £20bn without saving any energy. Here are papers on the technical security and security economics of smart meters, on their privacy, and on their deployment.
- On the Reliability of Electronic Payment Systems describes work I did to help develop prepayment utility metering, which made possible the electrification of millions of homes in less developed countries. The STS standard we developed is now used in 400m meters in over 100 countries.
Economics, psychology and criminology of information security
Incentives matter as much as technology for the security of large-scale systems.
Systems break when the people who could fix them are not the people who suffer
the costs of failure. So it's not enough for security engineers to understand
cryptomathematics and the theory of operating systems; we have to understand
game theory and microeconomics too. I pioneered the discipline of security economics which
is starting to embrace privacy economics, security psychology and criminology too.
- Silicon Den: Cybercrime is Entrepreneurship analyses underground criminal enterprises as tech startups; their main impediment compared with regular tech businesses may be lack of access to finance (blog)
- The gift of the gab: Are rental scammer skilled at the art of persuasion? studies accommodation frauds perpetrated against Cambridge students and postdocs. The fraudsters use standard boilerhouse sales techniques, and succeed because law enforcement ignore them (blog).
- Bitcoin Redux explains what’s gone wrong in the world of cryptocurrencies (blog). It follows on from Making Bitcoin Legal, which describes a better way of tracing stolen bitcoin (blog video).
- Taking Down Websites to Prevent Crime analyses the takedown industry. Private firms are better at taking down websites than the police; they do a lot more of it!
Reconciling Multiple Objectives –- Politics or Markets? discusses how institutional economics can help explain how protocols evolve (blog).
Lying Feels the Right Thing to Do reports that people are more likely to lie
when they feel rejected (blog blog press).
- It’s All Over but the Crying: The Emotional and Financial Impact of Internet Fraud shows that fraud victims suffer significant emotional harm as well as financial loss (blog followup)
freeze or not to freeze shows how you may be able to build a better lie
detector by analysing body motion, while Mining
Bodily Cues to Deception, analyses the signals that can be extracted
from different limb movements (blog Guardian Mail).
Measurement of Attitudes Regarding Cybercrime discusses how prosecutors and
public opinion are out of step; the former consider protest crimes to be more
serious than crimes done for financial gain, while voters take the opposite
- We will make
you like our research: the development of a susceptibility-to-persuasion
scale presents a questionnaire for determining how
gullible fraud victims are, and indeed how vulnerable people are in general to
manipulation by marketers (SSRN blog).
- Reading this
may harm your computer – The psychology of malware warnings analyses
what sort of text we should put in a warning if we actually want the user to
pay attention to it (blog).
the Cost of Cybercrime sets out to debunk the scaremongering around online
crime that governments and defence contractors are using to justify everything
from increased surveillance to preparations for cyberwar. It was written in
response to a request from the UK Ministry of Defence, and appeared at WEIS 2012 (press: BBC PC
- We wrote a major report for ENISA on the
Resilience of the Internet interconnection ecosystem which has been
adopted as ENISA policy. We believe this is the first time anyone has
documented how the Internet actually works in practice, as opposed to in theory.
This link will
take you to both the full report (238 pages) and the 31-page executive
- Tyler Moore and I wrote a series of survey papers on security economics as
research in the field got going. The 2011 tech
and Internet Security: a Survey of Recent Analytical, Empirical and Behavioral
Research appeared later as
a book chapter. An
Security Economics – and Beyond, appeared in various versions from
2006 to 2009. There was
a short survey
in Science in late 2006;
for economists at Softint in January 2007;
a version for
security engineers at Crypto in August 2007
chapter for mathematicians; and finally an archival journal version
Trans Roy Soc A (Aug 2009).
Economics – A Personal Perspective is an invited talk I gave at ACSAC
2012 telling the history of security economics slides).
- It's the
Anthropology, Stupid! discusses how we might put context and emotion back
into security decisions.
- The Economics of
Online Crime appeared in the Journal of Economic Perspectives; it looks at
the econometrics of fraud and phishing, and makes a number of suggestions for
improving the responses of banks and law-enforcement agencies.
- The Impact
of Incentives on Notice and Take-down examines how take-down speed varies
as a function of incentives. Banks are quick to remove phishing websites that
mention them by name, but they ignore mule recruitment websites because it's
harder to tell which bank will be affected.
- We have two futher papers on security economics in banking. The first is
on Verified by
VISA – the mechanism that asks for your card password when you shop
online. This is an example of how a poor design can win out if it has strong
deployment incentives (see
post and slides).
The second, On
the Security of Internet Banking in South Korea, analyses the effects of
Korea's decision to use national cryptography standards for Internet banking
rather than just using the same protocols as the rest of the world.
- On the
security economics of electricity metering appeared at WEIS 2010 and warns
that the government's smart meter programme probably won't work. Other papers
on security economics and control systems include Security Economics
and Critical National Infrastructure (at WEIS 2009); Certification and
Evaluation (at IEEE ETFA
2009); The Protection
of Substation Communications (SCADA Security Scientific Symposium, 2010);
a security architecture for substations (IEEE PES – ISGT Europe,
- The Trust Economy
of Brief Encounters argues that as transactions become more transient, we
will have to authenticate more; it appeared at the protocols workshop in 2009.
- We did a major study of Security Economics in the Single Market for the European Network
and Information Security Agency. We looked at the market failures underlying
spam, phishing and other online problems, and made concrete policy proposals,
some of which have been adopted. A shorter
version (62 pages) appeared at WEIS 2008 (slides)
and an even
shorter version (25 pages), at ISSE 2008.
- Closing the
Phishing Hole – Fraud, Risk and Nonbanks reports research on payment
regulation commissioned by the US Federal Reserve. This paper identified speedy
asset recovery as the best way to deter online fraud and rapid, irrevocable
payment instruments (such as Western Union) as a systemic threat.
- Why Information
Security is Hard – An Economic Perspective was the paper that got
information security people thinking about economics. It applies microeconomic
analysis to explain many phenomena that security folks had found to be
pervasive but perplexing.
- My `Trusted
Computing' FAQ undermined the Trusted Computing Group's initiative to
install DRM hardware in every computer, PDA and mobile phone. `TC' was sold to
Hollywood as a DRM platform but its real beneficiary would have been the
software industry whose customers would have been locked in more tightly. Cryptography and
Competition Policy – Issues with `Trusted Computing' is an economic
analysis I gave at WEIS2003 and as an
invited talk at PODC 2003. A short version
of the paper appeared in Cepis Upgrade). I spoke about TC at the "Trusted Computing Group" Symposium, which helped
drive German and EU policy. The row was ignited by a paper on the security
of free and open source software I gave at Softint 2002; see coverage in the
New York Times and The Register.
- In my paper on the security
of free and open source software, I show that the old argument whether
source code access makes it easier for the defenders to
find and fix bugs, or easier
for the attackers to find and exploit them is misdirected. Under standard
assumptions used in reliability growth modelling, the two will exactly cancel
each other out. That means that whether open or closed systems are more secure
in a given situation will depend on whether, and how, the application deviates
from the standard assumptions. These ideas are developed further in Open and Closed
Systems are Equivalent (that is, in an ideal world) which appeared as a
chapter in Perspectives
on Free and Open Source Software. See press coverage in slashdot, news.com and The
- On Dealing with
Adversaries Fairly applies election theory (also known as social choice
theory) to the problem of shared control in distributed systems.
- The Economics
of Censorship Resistance examines when it is better for defenders to
aggregate or disperse. Should file-sharers build one huge system like gnutella
and hope for safety in numbers, or should everyone just share the stuff they
care about? More generally, what are the tradeoffs between diversity and
solidarity when conflict threatens? (This was starting to be a
topic in social policy, and has led to a lot of research since.) Our
paper appeared at WEIS 2004.
- Here are papers on The Initial Costs
and Maintenance Costs of Protocols, which appeared at Security Protocols
2005, and How
Much is Location Privacy Worth? from WEIS 2005.
There are two relevant workshops I helped establish:
Security and Human Behaviour
workshop which brings together security engineers and psychologists, while
the Workshop on Economics and
Information Security is where you meet everyone working in security economics.
Peer-to-Peer and social network systems
One of the seminal papers in peer-to-peer systems
Eternity Service, which I invented in response to
censorship, The modern era only started once the printing press enabled
seditious thoughts to be spread too widely to ban. But when books no longer
exist as tens of thousands of paper copies, but as a file on a single server,
will courts be able to order them unpublished once more? (This has
to newspaper archives in Britain.) So I invented the Eternity Service as a
means of putting electronic documents beyond the censor's grasp. It inspired
second-generation censorship-resistant systems such
and Freenet; one descendant
is wikileaks. But the killer app turned
out to be not sedition, or even pornography, but copyright. Hollywood's action
against Napster led to our
ideas being adopted in filesharing systems; they are now re-emerging in the Internet of Things.
Work since the Eternity paper includes the following.
You Believe in Tinker Bell? The Social Externalities of Trust explores how
we can crowdsource trust. Just as a religion's power comes from its faithful
rather than from the government, so also a trust service's power should derive
from the users who trust it, rather than from a CA that's too big to fail (blog)
Experimental Evaluation of Robustness of Networks studies the best attack
and defence strategies in different kinds of networks. It builds on an earlier
topology of covert conflict, which asked how the police can best target an
underground organisation given some knowledge of its patterns of communication,
and how might they in turn might react to various law-enforcement strategies.
Authentication – harder than it looks shows how Facebook's social
captcha system is vulnerable to guessing by friends and to face recognition
Node Centrality in Complex Networks proposes new metrics for analysing
highly dynamic systems. If there's an epidemic of flu, should you close down the
schools or the subway? (blog news)
Prediction in Dynamic Human Contact Networks examines empirical methods for
predicting centrality of individuals in different contact networks that evolve
- Eight Friends
are Enough: Social Graph Approximation via Public Listings shows how easy it
is for an outsider to work out the structure of friendships on Facebook (see our
blog on Facebook's technical privacy and its democracy theatre.)
New Strategies for
Revocation in Ad-Hoc Networks analyses when it makes economic sense to use
suicide bombing as a tactic. Suicide attacks are found widely in nature, from
bees to helper T-cells; this model may help explain why (press
The idea was developed further
in Fast exclusion
of errant devices from vehicular networks.
- The Resurrecting
Duckling: Security Issues for Ad-hoc Wireless Networks describes how to do
key management between low-cost devices without either the costs or privacy
problems of central servers: trust on first use, followed by a factory reset if
need be. (There's also a journal
- The trust-on-first-use technique was first used at scale
Infection – Smart trust for Smart Dust applied the approach to ad-hoc
networks. Peers establish keys opportunistically, and you work out how to
recover from later node compromise.
AV is an industry standard I helped design for broadband communication over
the power mains, which is widely used in wireless LAN extenders. It also uses
trust-on-first-use key management, and the critical problem turned out to be:
how do you recover if you don't recruit the right device, but a similar one
DHT routing shows how we can make peer-to-peer systems more robust against
disruptive attacks if we know which nodes introduced which other nodes.
- The Economics of
Censorship Resistance examines when it is better for defenders to aggregate
or disperse. Should file-sharers build one huge system like gnutella and hope
for safety in numbers, or would a loose federation of fan clubs for different
bands work better?
- A keynote talk about next-generation
peer-to-peer systems at Wizards of OS 2004
discussed how Usenet might be reimplemented.
- A New Family of
Authentication Protocols presented our "Guy Fawkes Protocol", which lets
users sign messages using only two computations of a hash function and one
reference to a timestamping service. It led to the Tesla research on protocols
for signing digital streams.
- The Cocaine
Auction Protocol explored how transactions can be conducted between
mutually mistrustful principals with no trusted arbitrator, while giving a high
degree of privacy against traffic analysis.
- The Eternal
Resource Locator: An Alternative Means of Establishing Trust on the World Wide
Web investigated how to embed trust mechanisms in html documents. It grew
out of a medical school project to protect drug data; for details, see Secure Books:
Protecting the Distribution of Knowledge. We also looked at
how to secure a digital repository. This evolved into Jikzi, an
service which also caches links on which you've relied.
- The XenoService
– A Distributed Defeat for Distributed Denial of Service described
defeating DDoS attacks using a network of web hosts that can respond to an
attack on a site by replicating it rapidly and widely. It used Xen, a
hypervisor developed at Cambridge for distributed hosting, which led to
Reliability of security systems
I have been interested for many years in how security systems fail in real
life; many security designs are poor because they are based on unrealistic
threat models. I started with a study of ATM fraud, and expanded to other
applications one after another. This provides a central theme of
my book. I also have a
separate page on bank
security which gathers together all our papers on fraud in payment systems.
- Making Security Sustainable discusses a new challenge: how we will manage to continue patching cars and other safety-critical durable goods for decades (blog)
- What you get is what you C: Controlling side effects in mainstream C compilers shows how our toolsmiths could be our allies rather than a subversive fifth column in our rear (blog).
- Standardisation and Certification in the Internet of Things discusses what happens when we get software everywhere. Security will be more and more about safety. There will be many fasincating engineering challenges. The paper is a short version of a big report we did for the Europen Commission on the future of safety regulation – which will need a serious rethink! (short video longer video Prospect blog)
- International Comparison of Bank Fraud Reimbursement is a comparative study of the
security advice banks give their customers, of whether customers understand it,
and whether they think it fair (blog press)
- Our Security Analysis of
Factory Resets shows that in most Android phones, the factory reset function
doesn't work very well; it's usually possible to recover credentials for gmail
and other services along with personal data. Our Security Analysis of
Consumer-Grade Anti-Theft Solutions Provided by Android Mobile Anti-Virus
Apps shows that third-party security offerings are no better (blog The Verge Ars Technica Register BBC).
with the enemy on network management describes a project to develop a
version of Quagga for software defined networking research. It
appeared at Security Protocols 2014. Authentication
for Resilience: the Case of SDN discusses the authentication problems we
need to solve, and appeared at Security Protocols 2013.
- Rendezvous is a
prototype search engine for code, which recasts decompilation as a search
- Be Prepared: The EMV Pre-play Attack discloses a family of vulnerabilities in
EMV, the protocol underlying Chip and PIN payments. This may explain many
disputed transactions that look like card cloning and which the banks often
refuse to refund to fraud victims (blog BBC FT PCW Schneier) conference version of paper
Certification Systems Fail: Lessons from the Ware Report analyses failures
in the Common Criteria, FIPS 140 and other certification mechanisms by studying
lessons from the banking industry.
a research platform deconflating hardware virtualization and protection is
the first paper on a large
project we have with SRI to build a CPU supporting capabilities, port
FreeBSD to it, and build some demonstrator apps exploring the costs and
benefits of CPUs with hardware support for more fine-grained access control.
Practical Policy Enforcement for Android Applications describes how to
repackage Android apps to add user sandboxing and policy-enforcement code (source code).
and privacy implications of consumer payment innovation discusses what
threats to competition, privacy and payment security might arise as a result of
mobile innovation; I gave it at the Fed's biennial Payment
Systems Conference (slides blog).
birthday present every eleven wallets? is the first proper study of the
security of customer-selected bank PINs, and documents all sorts of bad stuff
(blog, press, blog).
controls the off switch? describes the strategic vulnerability created by
the UK plan to replace 47m gas and electricity meters with ‘smart
meters’ that can be switched off remotely. There are further papers on
SCADA security engineering here, here, here, here and here.
- A recurrring theme is the vulnerabilities in the EMV payment system, known
in the UK as Chip
and PIN. We won an award
for a paper
describing a man-in-the-middle attack that allows a stolen card to be used with
any pin. There was a TV
see also ZDnet,
the Register, Bruce
Schneier, the press
Rather than fixing the problem, the UK banks sought
our research; see comment in
News, Slashdot, Ars
- Other recent work on problems with bank systems includes Can We Fix the
Security Economics of Federated Authentication? which explores how we can
deal with a world in which your mobile phone contains your credit cards, your
driving license and even your car key (blog); a paper on whether
EMV is bad for innovation; a paper on Verified by
VISA, the mechanism that asks for your card password when you shop online;
and a tech report On
the Security of Internet Banking in South Korea.
- Optimised to
Fail: Card Readers for Online Banking documents the shortcomings of the
CAP card readers used for online banking; see also our blog, press
coverage and the later journal
inside the box: system-level failures of tamper proofing documented serious
vulnerabilities in Chip and PIN payment terminals and won the Best Practical Paper award at
the 2008 Oakland
conference. It was also featured on Newsnight. Here are some frequently
asked questions, our press
release, and coverage in the Register,
blog and the Telegraph. My paper Failures on
Fraud appeared in a central bankers' magazine and argued that all this is
yet another symptom of the failure of bank regulation.
snooping dragon: social-malware surveillance of the Tibetan movement
explains how the Chinese intelligence services compromised many of the
computers at the Dаlai Lаma's private office, and what this means for
information security (also slides).
- Why Cryptosystems
Fail was my first widely-cited paper and the first on what goes wrong with
payment systems. This version appeared at ACMCCS 93 and explains how ATM fraud
was done in the early 1990s.
Computer Security – Nine Principles took this work further, and
examines the problems with relying on cryptographic evidence. The recent
introduction of EMV ('chip and PIN') was supposed to fix the problem, but
and Chips documents protocol weaknesses in EMV,
and A Note on EMV
Secure Messaging in the IBM 4758 CCA documents even
Man-in-the-Middle Defence shows how to turn protocol weaknesses to
advantage. See my
paper RFID and
the Middleman for the likely next wave of frauds.
- On a New Way to
Read Data from Memory describes techniques we developed that use lasers to
read out memory contents directly from a chip, without using the read-out
circuits provided by the vendor. The work builds on methods described in Optical Fault
Induction Attacks, which showed how laser pulses could be used to induce
faults in smartcards that would leak secret information. That paper appeared at
CHES 2002; it made the
front page of the New York
Times and also got covered by slashdot.
It led to the field of semi-invasive attacks on semiconductors, pioneered by my
then research student Sergei Skorobogatov.
- After we discovered the above attacks, we developed a CPU technology that
uses redundant failure-evident logic to thwart attacks based on fault induction
or power analysis. Our
on this technology won an award at Async 2002. Our journal
Self-Checking Asynchronous Logic for Smart Card Applications, has more.
- Our classic paper on hardware security, Tamper Resistance
– A Cautionary Note, describes how to penetrate the smartcards and
secure microcontrollers of the mid-1990s. It kicked off the modern academic
study of hardware security and won a Best Paper award. Our second paper on the
subject was Low Cost Attacks on
Tamper Resistant Devices, which describes a number of further tricks. See
also the home page of our hardware security
laboratory, and Markus Kuhn's page of links to hardware
- On the
Reliability of Electronic Payment Systems describes work I did to help
develop prepayment utility metering, which made possible the electrification of
millions of homes in Africa. It appeared in the May 1996 issue of the IEEE
Transactions on Software Engineering. An ealier version, entitled Cryptographic
Credit Control in Pre-Payment Metering Systems, appeared at Oakland 95. A
later paper on this
subject discussed how we could apply what we'd learned to support utility
meter interworking in the UK after deregulation.
- On the Security
of Digital Tachographs successfully predicted how the introduction of
smartcard-based digital tachographs throughout Europe from 2005 would affect
fraud and tampering.
- How to Cheat
at the Lottery reports a novel and, I hope, entertaining experiment in
software requirements engineering.
- The Grenade
Timer describes a novel way to protect low-cost processors against
denial-of-service attacks, by limiting the number of cycles an application can
- The Millennium Bug
– Reasons Not to Panic describes our experience in coping with the
bug at Cambridge University and elsewhere. This paper correctly predicted that
the bug wouldn't bite very hard. Journalists were not interested. I later
discussed what we could learn from the incident in
interview with Stephen Fry.
Memorability and Security of Passwords – Some Empirical Results
tackles an old problem – how do you train users to choose passwords that
are easy to remember but hard to guess? We did a randomized controlled trial
with a few hundred first year science students which confirmed some folk
beliefs, but debunked some others. This became one of the classic papers on
law, the fitness of evolving species, and the limits of software
reliability applies the techniques of statistical thermodynamics to the
failure modes of any complex system that evolves under testing. The resulting
reliability growth model is in close agreement with empirical data, and
work in security economics.
Policies play a central role in secure systems engineering. They provide a
concise statement of the kind of protection a system is supposed to achieve.
This article is a security policy tutorial.
cryptography with biometrics shows that in those applications where you can
benefit from biometrics, you often don't need a large central database (as
proposed for Britain's ID card). There are
smarter and less privacy-invasive ways to arrange things.
The papers on physical security by Roger Johnston's team are
also definitely worth a look; see also an old leaked copy of the NSA Security Manual.
Robustness of cryptographic protocols
Many security system failures are due to poorly designed protocols, and this
has been a Cambridge interest for many years. Some relevant papers follow.
- CoverDrop: Securing Initial Contact for Whistleblowers is a better way for a newspaper to help anonymous sources get in touch. By hiding traffic in the paper's own app, it prevents the traffic-analysis attacks that are possible against users of SecureDrop and Signal.
- DigiTally: Piloting Offline Payments for Phones reports a field trial of a system we designed to extend mobile phone payments to places with no phone service. The protocol design itself is described in SMAPs: Short Message Authentication Protocols (blog slides discussion).
- API Level
Attacks on Embedded Systems are a powerful way to attack cryptographic
processors, and indeed any systems where more trusted processes talk to less
trusted ones. We found that a "secure" device can often be defeated by sending
it some sequence of transactions which its designer did not expect. We've
defeated pretty well every security processor we've looked at, at least once.
This line of research started at Protocols 2000 with The Correctness of
Crypto Transaction Sets; more followed in the first edition of
my book. Robbing
the bank with a theorem prover shows how to apply advanced tools to the
problem, and ideas for future research can be found in Protocol
Analysis, Composability and Computation. For a snapshot of how this
interacts with physical security, see
our survey of
cryptographic processors, a shortened version of which appeared in the
February 2006 Proceedings of the IEEE. An up-to-date survey of API attacks can
be found in the second edition of
book. There is also an API security
FAQ and an annual
protocols and evidence: where many payment systems fail analyses why
dispute resolution is hard. In a nutshell, the systems needed to support it
properly just don't get built (blog).
for Resilience: the Case of SDN discusses the authentication problems we
need to solve if we're to move software defined networks out of the data centre
into more heterogeneous environments. It appeared at Security Protocols 2013.
- Can We Fix the
Security Economics of Federated Authentication? explores how protocols work,
or fail, at global scale. How can we deal with a world in which your mobile
phone contains your credit cards, your driving license and even your car key
– and in particular what happens when it gets stolen or infected? (blog)
Management for Substations: Symmetric Keys, Public Keys or No Keys? debunks
the proposal to mandate public-key crypto in electricity substations. In this
particular application, the right solution is usually to have no crypto at all.
- What Next
after Anonymity? argues that it isn't enough to worry about the
confidentiality of metadata (anonymity); we sometimes need to protect their
integrity as well.
Satan's Computer is a phrase Roger Needham and I coined to express
the difficulty of designing cryptographic protocols; it has recently been
popularised by Bruce Schneier (see, for example, his foreword to my book). The problem of
designing programs which run robustly on a network containing a malicious
adversary is rather like trying to program a computer which gives subtly wrong
answers at the worst possible moment.
principles for public key protocols gives a number of attacks on protocols
based on public key primitives. It also puts forward some principles which can
help us to design robust protocols, and to find attacks on other people's
designs. It appeared at Crypto 95.
- The Cocaine
Auction Protocol explores how transactions can be conducted between
mutually mistrustful principals with no trusted arbitrator, even in
environments where anonymous communications make most of the principals
- The Initial
Costs and Maintenance Costs of Protocols appeared at the 2005 Protocols
Workshop and shows how economics can enter into protocol design.
- NetCard - A
Practical Electronic Cash Scheme presents research on micropayment
protocols for use in electronic commerce. We invented tick payments
simultaneously with Torben Pedersen and with Ron Rivest and Adi Shamir; we all
presented our work at Protocols 96.
- The GCHQ
Protocol and its Problems pointed out a number of flaws in a key management
protocol promoted by GCHQ as a European alternative to Clipper, until we shot
it down with this paper at Eurocrypt 97. Many of the criticisms we developed
here also apply to the more recent, pairing-based cryptosystems.
- The Formal
Verification of a Payment System describes the first use of formal methods
to verify an actual payment protocol, which was (and still is) used in an
electronic purse product (VISA's COPAC card). This is a teaching example I use
to get the ideas of the BAN logic across to undergraduates. There is further
detailed information in a technical
report, which combines papers given at ESORICS 92 and Cardis 94.
Attack on Server Assisted Authentication Protocols appeared in Electronics
Letters in 1992. It breaks a digital signature protocol.
- On Fortifying
Key Negotiation Schemes with Poorly Chosen Passwords presents a simple way
of achieving the same result as protocols such as EKE, namely preventing
middleperson attacks on Diffie-Hellman key exchange between two people whose
shared secret could be guessed by the enemy.
Protocols have been the stuff of high drama. Citibank asked the High Court to
disclosure of certain crypto API
vulnerabilities that affect a number of systems used in banking. I wrote to
the judge opposing
this; a gagging
order was still imposed, although in slightly less severe terms than
Citibank had requested. The trial was in camera, the banks' witnesses didn't
have to answer questions about vulnerabilities, and new information revealed
about these vulnerabilities in the course of the trial may not be disclosed in
England or Wales. Information already in the public
domain was unaffected. The vulnerabilities were discovered by Mike Bond and me while acting as the
defence experts in a phantom withdrawal court case, and independently discovered
by the other side's expert, Jolyon Clulow, who later joined us as
a research student. They are of significant scientific interest, as well as being
relevant to the rights of the growing number of people who suffer phantom withdrawals from their bank
accounts worldwide. Undermining the fairness of trials and forbidding discussion
of vulnerabilities isn't the way forward (press coverage by the
Cryptography, including quantum cryptography
Lots of people don't believe quantum crypto is practical. I also don't
believe the security proofs offered for entanglement-based quantum
cryptosystems, because they assume that the strange behaviour observed
in the Bell tests must result from nonlocal action. But it can also
emerge from pre-existing long-range order. One explanation, advocated
by Nobel prizewinner Gerard 't Hooft, is the cellular automaton
interpretation of quantum mechanics; see his keynote talk
2015. I have done some work with
Robert Brady to develop another
line of inquiry.
Maxwell's fluid model of
magnetism shows that a wavepacket travelling along a phase vortex
in an Eulerian fluid obeys Maxwell's equations, is emitted and
absorbed discretely, and can have linear or circular
polarisation. What's more, the measured correlation between the
polarisation of two cogenerated wavepackets is exactly the same as
predicted by quantum mechanics, and observed in the Bell tests
- If you're new to this subject, a good starting point is to watch the video of Yves Couder's
beautiful bouncing-droplet experiments, and then read our paper Why bouncing droplets are a pretty good
model of quantum mechanics. This shows how droplets bouncing on a vibrating
fluid bath obey two-dimensional analogues of Maxwell's equations and a version
of Schrödinger's equation.
- For the hard math, which explains how fermionic quasiparticles obeying
Dirac's equation can arise in a bosonic fluid, see this paper; another paper that may be
relevant is here. And here's a video of my talk at the 2015 Crossing
In the 1990s I worked with Eli Biham and Lars Knudsen to develop Serpent – a
candidate block cipher for the Advanced
Encryption Standard. Serpent got the second largest number of votes.
Other papers on cryptography and cryptanalysis include the following.
- The Dancing Bear
– A New Way of Composing Ciphers presents a new way to combine crypto
primitives. Previously, to decrypt using (say) any three out of five keys, the
keys all had to be of the same type (such as RSA keys). With my new
construction, you can mix and match - RSA, AES, even one-time pad. The paper
appeared at the 2004 Protocols Workshop; an earlier version came out at the FSE 2004 rump session.
- Two Remarks on
Public Key Cryptology is a note on two ideas I floated at talks I gave in
1997-98, concerning forward-secure signatures and compatible weak keys. The
first of these has inspired later research by others; the second gives a new
attack on public key encryption.
Practical and Provably Secure Block Ciphers: BEAR and LION shows how to
construct a block cipher from a stream cipher and a hash function. We had
already known how to construct stream ciphers and hash functions from block
ciphers, and hash functions from stream ciphers; so this paper completed the
set of elementary reductions. It also led to the "Dancing Bear" above.
- Tiger –
A Fast New Hash Function defines a new hash function, which we designed
following Hans Dobbertin's attack on MD4. This was designed to run extremely
fast on the new 64-bit processors such as DEC Alpha and IA64, while still
running reasonably quickly on existing hardware such as Intel 80486 and
Pentium (the above link is to the Tiger home page, maintained in Haifa by Eli
Biham; if the network is slow, see my UK mirrors of the Tiger paper, new and old reference
implementations (the change fixes a padding bug) and S-box generation
documents. There are also third-party crypto toolkits supporting Tiger,
such as that from Bouncy Castle).
- Minding your
p's and q's points out a number of things that can go wrong with the choice
of modulus and generator in public key systems based on discrete log. It
elucidated some of the previously classified reasoning behind the design of the
US Digital Signature Algorithm, and appeared at Asiacrypt 96.
– A New Kind of Stream Cipher shows how to do traitor tracing using
symmetric rather than public-key cryptology. The idea is to turn a stream
cipher into one with reduced key diffusion, but without compromising
security. A single broadcast ciphertext is decrypted to slightly different
plaintexts by users with slightly different keys. This paper appeared
at Fast Software
Encryption in Haifa in January 1997.
for the Optimum Correlation Attack shows that nonlinear combining functions
used in nonlinear filter generators can react with shifted copies of themselves
in a way that opens up a new and powerful attack on many cipher systems. It
appeared at the second workshop on fast software encryption.
- The Classification of
Hash Functions showed that correlation freedom is strictly stronger than
collision freedom, and shows that there are many pseudorandomness properties
other than collision freedom which hash functions may need. It appeared at
Cryptography and Coding 93.
- A Faster Attack
on Certain Stream Ciphers shows how to break the multiplex shift register
generator, which is used in satellite TV systems. I found a simple
divide-and-conquer attack on this system in the mid 1980's, a discovery that
got me "hooked" on cryptology. This paper is a refinement of that work.
- On Fibonacci
Keystream Generators appeared at FSE3, and shows how to break "FISH", a
stream cipher proposed by Siemens. It also proposes an improved cipher, "PIKE",
based on the same general mechanisms.
- Tree Functions and
Cipher Systems appeared in 1991; it points out a weakness in a proprietary
cipher that was later developed into this.
Another of my contributions was founding the series of workshops on Fast Software Encryption.
Security of Clinical Information Systems
The safety and privacy of clinical systems have been a problem for
years. Recent scandals include
DeepMind case (exposed by my then postdoc Julia Powles) where the
Royal Free Hospital gave Google a million patients' records that they
shouldn't have; and
affair where a billion records – basically all hospital care
episodes since 1998 –
to 1200 firms worldwide, in a format that enabled many patients to
be re-identified. It
wasn't much better under the previous Labour government, which had
over thoughtless and wasteful centralisation. There is now an
which monitors and campaigns for health privacy.
The NHS has a
Gordon Brown's own medical records were compromised while he was prime
minister, but the miscreant got
as it was "not in the public interest" to prosecute him. In another
famous case, Helen Wilkinson had to organise
in Parliament to get ministers to agree to remove defamatory and
untrue information about her from NHS computers. The minister assured
the House that the libels had been removed; months later, they still
had not been. There is now an NGO set up specifically
to campaign for health privacy, medConfidential.org.
Here are my most recent papers on the subject.
in Remote Clinical Practice is a report I wrote for the International
Psychoanalytical Association, analysing what we learned before and during the
pandemic on the safety and privacy of remote psychotherapy.
collection, linking and use of data in biomedical research and health care:
ethical issues is a report we wrote for the Nuffield Bioethics Foundation:
what happens to health privacy in a world with cloud-based medical records and
pervasive genomics? (blog Guardian Indy Press Association Science)
State is a report we wrote for the Joseph Rowntree Reform Trust on
the failings of public-sector IT in Britain, and how to fix them. It pointed
out that a number of health systems almost certainly break European law. There's
coverage on the BBC, in the Guardian (also here), the Mail (also
the Independent, the Telegraph, E-Health Insider and Liberty Central. This report had a lot of impact; the coalition
government promised to abolish or at least change a number of the systems we
fingered as unlawful. Both the Conservatives and the Lib Dems promised to axe
the NHS centralisation project too if they won the 2010 election; after they did
so, the name was changed but the stupidity continued.
- In 2006 I organised 23 computer science professors to write to the Health
Committee requesting an independent review of the
NHS National Programme for IT, the last big centralisation drive, as it was
visible failing. Ministers refused, and NPfIT
went on to become the largest civil-government IT project failure ever.
- I was one of the authors of a 2006 report on the safety and
privacy of children's databases, done for the UK Information Commissioner.
It concluded that government plans to link up most of the public-sector
databases that hold information on children were misguided: the proposed systems
would be both unsafe and illegal. This report got a lot of
publicity. I spoke on these issues on these
videos made by Action on Rights for Children.
- I wrote a report
for the National Audit Office on the health IT expenditure, strategies and
goals of the UK and a number of other developed countries. This showed that the
NHS National Program for IT is in many ways an outlier, and high-risk.
- Here is an article I
wrote for Drugs and Alcohol Today analysing the likely effects of the NHS
computing project on patient privacy, particularly in the rehabilitation field.
- In 2007 I acted as a Special Adviser to the House of Commons Health Select
on the Electronic Patient Record. (See also the parliamentary
debate on the report, press comment, and
an article on the implications
for HIV treatment.)
- Patient confidentiality
and central databases appeared in the February 2008 British Journal of
General Practice, calling on GPs to encourage patients to opt out of the NHS
care records service.
- System security for
cyborgs discusses technical, ethical and security-economics issues to do
with implantable medical devices.
Civil servants started pushing for online access to everyone's records in 1992
and I got involved in 1995, when I started consulting for the British Medical
Association on the safety and privacy of clinical information systems. Back
then, the police were given access to all drug prescriptions, after the
government argued that they needed it to catch doctors who misprescribed
heroin. The police got their data, but they didn't
catch Harold Shipman,
and no-one was held accountable. The NHS slogan in 1995 was `a unified electronic patient record, accessible to
all in the NHS'. The BMA campaigned against this, arguing that it would destroy
Security in Clinical Information Systems was published by the BMA in
January 1996. It sets out rules that uphold the principle of patient consent
independently of the details of specific systems. It was the medical
profession's initial response to the safety and privacy problems posed by
centralised NHS computer systems.
Update on the BMA Security Policy appeared in June 1996 and tells the story
of the struggle between the BMA and the government, including the origins and
development of the BMA security policy and guidelines.
- There are comments made
at NISSC 98 on the healthcare protection profiles being developed by NIST for
the DHHS to use in regulating health information systems privacy, which made a
number of mistaken assumptions about threats and protection mechanisms.
on the Caldicott Report raises a number of issues about the report of the
Caldicott Committee, which was set up by the Major government to kick the
medical privacy issue into touch until after the 1997 election. Its members
failed to understand that medical records from which the names have been
removed, but where NHS numbers remain, are not anonymous – as large
numbers of NHS staff need to map names to numbers in order to do their
technology in medical practice: safety and privacy lessons from the United
Kingdom provided an overview of the safety and privacy problems we
encountered in UK healthcare computing in the mid-90s for readers of the
Australian Medical Journal.
DeCODE Proposal for an Icelandic Health Database analyses a proposal to
collect all Icelanders' medical records into a single database. I evaluated
this for the Icelandic Medical Association and concluded that the proposed
security wouldn't work. The company running it soon hit financial
problems and later filed for bankruptcy. The ethical issues were a
Supreme Court allowed a woman to block access to her father's
records because of the information they may reveal about her (see analysis).
This effectively killed the vision of having the whole population on a database.
I also wrote an analysis
of security targets prepared under the Common Criteria for the evaluation of
this database. See also BMJ
correspondence and an article by Einar
System Security – Interim Guidelines appeared in the British Medical
Journal on 13th January 1996. It advises healthcare professionals on prudent
security measures for clinical data. The most common threat is that private
investigators use false-pretext telephone calls to elicit personal health
information from assistant staff.
Security Policy Model for Clinical Information Systems appeared at the 1996
IEEE Symposium on Security and Privacy. It presents the BMA policy model to the
computer security community in a format comparable to policies such as
Bell-LaPadula and Clark-Wilson. It had some influence on later US health
privacy legislation (the Kennedy-Kassebaum Bill, now HIPAA).
Wide Networking and Patient Confidentiality appeared in the British Medical
Journal in July 1995 and set out some early objections to the government's
health network proposals.
Confidentiality &ndash At Risk from NHS Wide Networking went into somewhat
more detail, particularly on the security policy aspects. It was presented at
Health Care 96.
with the NHS Cryptography Strategy points out a number of errors in, and
ethically unacceptable consequences of, a report on
cryptography produced for the Department of Health. These comments formed the
BMA's response to that report.
In 1996, the Government set up the Caldicott Committee to study the
matter. Their report
made clear that the NHS was already breaking confidentiality law by sharing
data without consent; but the next Government
and again) to
give itself the power to share health data as the Secretary of State saw
fit. (We objected
and pointed out
the problems the
bill could cause; similar sentiments were expressed in
a BMJ editorial,
and a Nuffield
analysis, and BMJ
and here. Ministers
claimed the records were needed for cancer registries: yet cancer researchers
work with anonymised data in other countries – see
There was a storm of protest in the press: see
the New Statesman,
Register. But that died down; the measure has now been consolidated
as sections 251
and 252 of the NHS Act 2006, the Thomas-Walport review blessed nonconsensual
access to health records (despite FIPR pointing out that this was
illegal – a view later supported by the European Court). A government
committee, the NHS Information Governmance
Board, was set up oversee this lawbreaking, and Dame Fiona is being wheeled out once more. Centralised,
nonconsensual health records not only contravene the I v Finland judgement but
of Helsinki on ethical principles for medical research and
the Council of
Europe recommendation no R(97)5 on the protection of medical data.
Two health IT papers by colleagues deserve special mention. Privacy in clinical
information systems in secondary care describes a hospital system
implementing something close to the BMA security policy (it is described in
more detail in a special issue of the Health
Informatics Journal, v 4 nos 3-4, Dec 1998, which I edited). Second, Protecting Doctors'
Identity in Drug Prescription Analysis describes a system designed to
de-identify prescription data for commercial use; although de-identification
usually does not protect patient privacy very well, there are exceptions, such
as here. This system led to a court case, in which the government tried to stop
its owner promoting it – as it would have competed with their (less
privacy-friendly) offerings. The government lost: the Court of Appeal decided
that personal health information can be used for research without patient
consent, so long as the de-identification is done competently.
Resources on what's happening in the USA include
many NGOs: Patient Privacy Rights may have
been the most influential, but see also EPIC, the Privacy Rights
Clearinghouse, the Citizens' Council on Health Care, the Institute for Health Freedom. and CDT. Older resources include
an NAS report entitled For the Record: Protecting
Electronic Health Information, a report by the Office of Technology
Assessment, a survey of the uses of
de-identified records for the DHHS, and a GAO report on their use
in Medicare. As for the basic science, see my book chapters on Boundaries and on Inference Control.
Public policy issues
I've been involved over the years with academic freedom, and with digital rights more generally.
Should our university require us to treat foolish or obnoxious colleagues with "respect", or just with "tolerance"? A free speech vote brought the culture wars to Cambridge, and we won. See Varsity, Newsweek, the FT, the Spectator, the Mail, the Sunday Times, the Times Higher Education Supplement, the Cambridge Student, the Cambridge Radical Feminist Network, Stephen Fry – and the Minister of State for Universities.
I chair the Foundation for Information Policy
Research, the UK's leading Internet policy think tank, which I helped set
up in 1998. We are not a lobby group; our enemy is ignorance rather than the
government of the day, and our mission is to understand IT policy issues and
explain them to policy makers and the press. Here's an overview of the issues as we saw them in 1999, and a video
of how we saw them ten years later in 2008. Some highlights of our work follow.
- Twenty Five Years of Crypto Wars:
Bugs in our Pockets: The Risks of Client-Side Scanning, a technical study of the risks involved in mandatory scanning of people's phones and other devices for illegal materials, as proposed in various forms by the US and UK governments, the EU and Apple (blog). This follows and updates a 2015 paper on the same topic, Keys Under Doormats, which
argues that the push by the UK and US governments for exceptional
access to all computer and communications data is wrong in principle and
unworkable in practice (see also this video and this followup).
In 2016, we organised the tenth Scrambling for Safety
workshop on their Investgatory Powers Bill while it was on its way through
Parliament. The chaos after the Brexit vote, plus May's appointment as Prime
Minister, allowed this bill to get through Parliament unscathed. The European
Court of Justice has
already found that its data retention provisions contravene human rights but
the government ignored this, and the Australian government followed
Protocol and its Problems pointed out a number of serious defects in
that the British government used to secure its electronic mail. Our analysis
stopped the protocol being more widely adopted; the government is still trying
to push its successor, which still
the same problems. The government also proposed mandatory licensing of
certification authorities, so we compiled The
Global Trust Register – a certification authority implemented in paper
and ink rather than electronics. Our book would have been banned by the new
law – which enabled us to visit Culture Secretary Chris Smith at a
critical point and get it on the Cabinet agenda.
What Goes Around Comes Around is a chapter I wrote for a book by EPIC, on whose advisory board I sit.
I first got engaged in technology policy thanks to attempts in the 1990s by
governments to control the use of cryptography. In 1995, I
wrote Crypto in
Europe – Markets, Law and Policy, the first paper to point out that
law enforcement communications intelligence was mostly about traffic analysis
and criminal communications security was mostly traffic security. The Risks
of Key Recovery, Key Escrow, and Trusted Third-Party Encryption became the
most widely-cited publication on key escrow; it was originally presented as
testimony to the US Senate, and then also to
and Industry Committee of the UK House of Commons, together with a further
piece I wrote, The
Risks and Costs of UK Escrow Policy.
What we achieved with this campaign was to limit the scope of
the Regulation of Investigatory Powers
Act. Originally this would have allowed the police to obtain, without
warrant, a complete history of everyone's web browsing activity, as
‘communications data’. Our ‘Big Browser amendment’got
the House of Lords to limit this to the identity of the machines involved in a
communication, rather than the full URLs. But the RIP Act still made it into
law and has had a number of
effects we predicted.
These issues revived in the 2000s with
Modernisation Programme, a plan to centralise all traffic data first in a
central database (under Blair and Brown) and then in a system of federated
databases maintained by communications service providers. FIPR wrote
various papers on related matters, and when the
Coalition Government brought its Communications
Data Bill, we organised resistance.
The bill was dropped after the Lib Dems finally vetoed it.
- Privacy has come under attack not just from the spooks but
from the world of Big Data.The
collection, linking and use of data in biomedical research and health care:
ethical issues is a report we wrote for the Nuffield Bioethics Foundation:
what happens to health privacy in a world with cloud-based medical records and
pervasive genomics? (blog Guardian Indy Science).
In 2009, our Database
State report on the failings of public-sector IT in Britain, and how to fix
them, got massive press coverage: the BBC, the Guardian (also here), the Mail (also
Central. This followed an
earlier ICO report on
children's databases. Both the Lib Dems and the Conservatives promised to
kill or change at least some of these systems; after they won power in the
2010 election their coalition agreement spelled the end of the ContactPoint
children's database, and of ID cards. The subsequent
review by my FIPR
colleague Eileen Munro also sealed the fate of eCAF, another central children's
- Sustainability interacts in various ways with information security, notably in the sustainability of software; but see also my talk on Privacy for Tigers.
- Brexit affects us in numerous ways. Brexit
and technology explained how the Brexit debate largely ignored network
externalities, which could make the damage worse. Brexit and
Cambridge assesses the likely costs to the University (blog posts).
- Waste of Public Money is another objection to the bad
government systems that undermine our privacy. Other wasteful systems include smart
meters which look set to cost billions without achieving anything useful (blog).
- Identity Cards were a clever political move by Blair; they
divided the Conservatives, so Blair promised to do them for almost a decade and never got round ot it.
to the Home Affairs committee in 2004 that they would not work as advertised,
and contributed to
Report that spelled this out in detail. I wrote various previous pieces in
response to government identity consultations, on aspects such
- Internet Censorship is a growing problem, and not just in
developing countries; I've been on the receiving end more than once. In 1995, I invented the first
censorship-resistant system, the Eternity
Service; this was a precursor of later file-sharing systems (see above), and we've also written on the economics of
censorship resistance. But despite the technical difficulties and
collateral costs of content filtering, governments aren't giving up. From 2006
to 2008, I was a principal investgator for the OpenNet Initiative which monitors
Internet filtering worldwide. Shifting
Borders reviewed the state of play in late 2007, and appeared in Index
on Censorship. Tools
and Technology of Internet Filtering goes into more technical detail. The
political action now is about Internet blocking.
Consumer Protection: FIPR also brought together legal and computing
experts to deconstruct the fashionable late-1990s notion that ‘digital
certificates’ would solve all the problems of e-commerce and e-government.
Anyone inclined to believe such nonsence should read Electronic
Commerce – Who Carries the Risk of Fraud?. Other work in this thread
include FIPR's responses to consultations on smartcards, the electronic signature
directive and the ecommerce
More recently we have seen the erosion of consumer rights as a result of the
introduction of chip and PIN cards. The technical sections above
describe how frauds happen; the flip side of the story is how the banks escape
liability. Our analysis of
the failings of the Financial Ombudsman Service remains unanswered; see
also FIPR's submission
on Personal Internet
Security (with which the House of Lords
and the National Payments
Plan. FIPR now takes
the view that the only way
to fix consumer protection is to replace public action with private action, by
changing the rules on costs so that consumers can enforce their rights in court
without risking horrendous costs orders if they lose.
- Export Control: In 2001-02, FIPR persuaded the Lords to
amend the Export
Control Bill. This bill was designed to give ministers the power to license
intangible exports. It was the result of US lobbying of Tony Blair in 1997;
back then, UK crypto researchers could put source code on our web pages while
our US colleagues weren't allowed to. In its original
form, its provisions were so broad that it would have given ministers the
power of pre-publication review of scientific papers. We defeated the Government
in the House of Lords by 150-108, following a hard campaign – see press
coverage in the BBC,
Scientist, the Guardian
and the Economist, and an article on free
speech I wrote for IEEE Computing. But the best quote I
have is also the earliest. The first book written on cryptology in English, by
Bishop John Wilkins in 1641, remarked that ‘If all those
useful Inventions that are liable to abuse, should therefore be concealed,
there is not any Art or Science which might be lawfully profest’
This issue revived in 2003, with a government attempt to wrest back by
regulation much of what they conceded in parliament. FIPR fought back
and extracted assurances
from Lord Sainsbury about the interpretation of regulations made under the
Act. Without our campaign, much scientific collaboration would have become
technically illegal, leaving scientists open to arbitrary harrassment. Much
credit goes to the Conservative frontbencher Doreen
Miller, Liberal Democrat frontbencher Margaret
Sharp, and the then President of the Royal Society Bob May,
who made his maiden speech in the Lords on the issue and marshalled the
crossbenchers. We are very grateful for their efforts.
- Trusted Computing was a focus in 2002-03. I wrote
a Trusted Computing
FAQ, followed by
a study of the
competition policy aspects which led inter alia to
organised by the German government that pushed the Trusted Computing Group into
incorporating. Microsoft couldn't get remote attestation to work; Intel abandoned
trusted computing; and its only direct descendants were bitlocker and Arm's
- IP Enforcement: Our lobbying priority in 2003-04 was the EU IPR enforcement
directive, which has been criticised by
distinguished lawyers. Our lobbying got it amended to remove
criminal sanctions for patent infringement and legal protection for devices such
as RFID tags. This law was supported by the music industry,
the luxury brands, and (initially) Microsoft, while the coalition that we
put together to oppose it included the phone companies, the supermarkets, the
generic drugmakers, the car parts industry, smaller software firms and the free
software community. The press was sceptical – in Britain, France and even America. The issue was even linked to a boycott of
Gillette. There is more on my blog.
This was a watershed in copyright history: the IP lobby was never going to
be stopped by fine words, only by another lobby pushing in the other direction,
and the Enforcement Directive was when that first came together. It also led to
the birth of EDRI, European Digital Rights, a
confederation of European digital-rights NGOs, whose establishment was one of
FIPR's significant achievements. EDRI's first campaign was against the IP
Enforcement Directive; afterwards FIPR and EDRI established
a common position on
intellectual property. Since then I have given evidence
to the Gowers Review of
committee on DRM. The lead UK NGO on IP nowadays is
the Open Rights Group.
- Terrorism: Here are Comments on Terrorism I
wrote after the 11th September attacks. The resulting hysteria made
economics to enable policymakers and others to think more rationally about
such things, once gthey calmed down. In the dark years that followed, I testified against police
attempts to increase pre-charge detention to ninety days; and here is
a video I did on the
effects of 9/11. We must
back on the scaremongers.
I served on Council, Cambridge University's governing body, 2003–10
and from 2015–18. I stood for election because of a
proposal that most of the intellectual property generated by faculty members
– from patents on bright ideas to books written up from lecture notes
– would belong to the university rather than to its creator. To stop
this, and to prevent further incidents
one), we founded
the Campaign for Cambridge
Freedoms. The final vote
approved a policy according to which academics keep copyright but the
University gets a share of patent royalties. I
got re-elected in
2006, and in my second term
we won an
to protect academic
freedom. For more, see
from the Oxford Magazine. From 2013-4 I was on
our Board of Scrutiny. In my third
term my main contribution was investigating
the delays and cost overruns in a large construction project.
My CV is here
while my h-index is tracked here.
I'm a Fellow of Churchill College, the Royal Society, the Royal Academy of Engineering, the Institution of Engineering and Technology, the
Institute of Mathematics and its
Applications, and the Institute of
Physics. I won the 2015
Lovelace medal; the interviews I did for that award are here, while my oral
history interview transcript is here and an Academy
video is here. As for my academic genealogy, my thesis adviser was Roger Needham; his was Maurice Wilkes; then it
runs back through Jack Ratcliffe, Edward Appleton,
JJ Thomson, Lord
Rayleigh, Edward Routh, William Hopkins, Adam Sedgwick, Thomas
Postlethwaite, Stephen Whisson, Walter
Smith, Roger Cotes, Isaac Newton, Isaac Barrow and Vincenzo Viviani to Galileo Galilei. For context, see
History of Cambridge University
Finally, here is my PGP
key. If I revoke this key, I will always be willing to explain why I have
done so provided that the giving of such an explanation is lawful. (For
more, see FIPR.)
The third edition is
now on sale – you can read sample chapters on my book page.
Security engineering is about building systems to remain dependable in the face
of malice, error or mischance. As a discipline, it focuses on the tools,
processes and methods needed to design, implement and test complete systems,
and to adapt existing systems as their environment evolves. My book has become
the standard textbook and reference since it was published in 2001. You can
download both the first and second editions without charge here; the third edition will become free from 2024.
Security engineering is not just concerned with infrastructure matters such as
firewalls and PKI. It's also about specific applications, such as banking and
medical record-keeping, and increasingly about embedded systems such as payment
terminals and burglar alarms. It's usually done badly, so it often takes
several attempts to get a design right. It's also hard to learn: although
there were good books on a number of the component technologies, such as
cryptography and operating systems, there was little about how to use them
effectively, and even less about how to make them work together. Most systems
don't fail because the mechanisms are weak, but because they're used wrong.
My book was an attempt to help the working engineer to do better. As well as
the basic science, it contains details of many applications – and lot of
case histories of how their protection failed. It describes a number of
technologies which aren't well described elsewhere. The first edition was
pivotal in founding the now-flourishing field
of information security
economics: I realised that the narrative had to do with incentives and
organisation at least as often as with the technology. The second edition
incoporated the economic perspectives we've developed since then, and new
perspectives from the psychology of security, as well as updating the
technological side of things. The third edition is an update for the new world
of phones, cloud services and social media; it tackles the problems raised by
cars and medical devices such as the interaction of security with safety, and
the costs of long-term patching; it also adds a huge amount about modern threat
actors, from the cybercrime ecosystem to what we learned about state
capabilities from the Snowden leaks and elsewhere.
Highlights by year
2020 highlights include sponge attacks and nudge attacks on machine-learning systems, along with work on adversarial reinforcement learning and on decoding smartphone sounds with a voice assistant. But my main project in 2020 was writing a third edition of my Security Engineering textbook.
2019 highlights include an acoustic side channel on smartphones, one paper on whistleblowing and two papers on blocking adversarial machine learning. The big paper was on Measuring the Changing Cost of Cybercrime; since we did the first systematic study seven years ago, the patterns changed surprisingly little despite a huge changed in technology. Finally I gave an invited talk at 36C3 on the sustainability of safety, security and privacy.
2018 highlights include papers on what's wrong with bitcoin exchanges and how to trace stolen bitcoin; on making security sustainable; controlling side effects in mainstream C compilers; how protocols evolve and a gullibility metric. There's also an invited talk on privacy for tigers.
2017 highlights include Standardisation and Certification in the Internet of Things, an analysis for the EU of what happens when we get software everywhere, and which informed EU directive 2019/771 on the sale of goods; DigiTally, a prototype payment system we built to extend mobile phone payments to areas of less developed countries with no phone service, using a novel protocol; and a book chapter I wrote for EPIC.
2016 highlights include a new Android side channel; an
investigation of the social externalities of trust; studies of when
lying feels the right thing to do, of taking down websites to prevent crime and bank fraud reimbursement; and finally two papers on Brexit.
2015 highlights included Keys Under Doormats, on
what's wrong with government attempts to mandate exceptional access to all our
data; a Nuffield report on what happens to health privacy in a world of
cloud-based medical records and pervasive genomics; a report on the
emotional impact of Internet fraud;
on how to do lie detection
using analysis of body motion; severe flaws in Android factory reset and
anti-virus apps; and a novel demonstration that the Bell test
results can come from pre-existing long-range order as easily as from nonlocal
2014 highlights included papers on Chip and
Skim describing pre-play frauds against EMV bank cards; Security
protocols and evidence which explains how the systems needed to support
proper dispute resolution just don't get built; Experimental
Measurement of Attitudes Regarding Cybercrime, on how prosecutors and public
opinion are out of step; The psychology
of malware warnings, on how to get users to pay attention to risk; Privacy
versus government surveillance, on network economics and international
relations; and Why bouncing droplets are a pretty good
model of quantum mechanics, which solves an outstanding mystery in physics.
2013 highlights included Rendezvous, a
prototype search engine for code; a demonstration that we could steal your PIN
via your phone camera
and microphone; an analysis of SDN
Authentication; and papers on quantum computing and Bell's inequality.
2012 highlights included a big report on Measuring
the Cost of Cybercrime and a history of
security economics; an attempt to kill the government's smart metering project; three
papers on dynamic
networks; and four papers on payment protocols: Chip and Skim: cloning
EMV cards with the pre-play attack, How
Certification Systems Fail, A
birthday present every eleven wallets? and Social
Authentication – harder than it looks. Finally, Risk
and privacy implications of consumer payment innovation discusses both
payment and economic issues.
2011 highlights included a major report on the Resilience
of the Internet Interconnection Ecosystem which studies how an attacker
might bring down the Internet; an updated survey paper on Economics and
Internet Security which covers recent analytical, empirical and behavioral
research; and Can We Fix the
Security Economics of Federated Authentication? which explores how we can
deal with a world in which your mobile phone contains your credit cards, your
driving license and even your car key. What happens when it gets stolen or
2010 highlights included a paper on why Chip
and PIN is broken for which we got coverage on Newsnight and a best
paper award (later, the banks tried to suppress
this research). Other bank security work included a paper on Verified by
VISA and another
on the unwisdom of banks adopting proprietary standards. On the control systems
front, we published papers on the technical
security and security
economics of smart meters, on their privacy, on their
deployment and on key management for
substations. I created a psychology and security web
page and wrote a paper on putting
context and emotion back in security decisions.
2009 highlights included Database
State, a report we wrote about the failings of public-sector IT – many
of whose recommendations were adopted by the government elected in 2010; The snooping
dragon which explains how the Chinese spooks hacked the Dalai Lama in the
run-up to the Peking Olympics; Eight Friends
are Enough, which shows how little privacy you have on Facebook; and The Economics of
Online Crime. There's a videos of a talk I gave on dependability at
as well as a survey paper,
and a podcast. Finally,
I wrote an Unauthorised
History of Cambridge University.
2008 highlights included
a major study of Security Economics
and European Policy for the European Commission; the second edition of my
Engineering"; the discovery
vulnerabilities in Chip and PIN payment systems;
an analysis of the failings
of the Financial Ombudsman Service (see also
a video from the World
Economic Forum in November 2008); the
FIPR submission to the
Thomas-Walport Review; a piece
in the British Journal of General Practice; three videos on
privacy made by ARCH; and a video on surveillance. I
started a Workshop
on Security and Human Behaviour to bring together psychologists with
economists and security engineers to work on deception and risk.
2007 highlights included technical papers
on RFID and
on New Strategies
for Revocation in Ad-Hoc Networks (which explores when suicide attacks are
effective); a paper on fraud, risk and
nonbank payment systems I wrote for the Fed; and a survey paper on Information
Security Economics (of which a shortened version appeared in Science). I was a
special adviser to House of Commons Health Committee for their Report
on the Electronic Patient Record. Finally, following the HMRC data loss, I
appeared in the debate on Newsnight.
highlights included technical papers on topics from protecting
power-line communications to the
Man-in-the-Middle Defence, as well as a major report on the safety and privacy of
children's databases for the Information Commissioner. I ended the year debating
health privacy with health minister Lord Warner.
highlights included research papers on The topology of
covert conflict, on combining
cryptography with biometrics, on Sybil-resistant DHT
routing, and on Robbing the bank
with a theorem prover; and a big survey paper on cryptographic
highlights included papers on cipher
establishment in ad-hoc networks and the economics of
censorship resistance. I also lobbied for amendments to the EU IP Enforcement
Directive and organised a workshop on copyright which led to a common position adopted by
many European NGOs.
I only write and referee for open publications, so I discard emails asking for reports
for journals that sit behind a paywall.
By default, when I post a paper here I license it under the relevant Creative Commons
license; you may redistribute it with attribution but not modify it.
I can no longer admit PhD students for Cambridge, because of forthcoming
mandatory retirement; so if you want to do a PhD, please read the relevant web pages. I still
admit PhD students at Edinburgh.