Ross Anderson

[Research] [Blog] [Videos] [Politics] [My Book] [Music] [Seminars] [Contact Details]

Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks shows that the randomness tests long used to check random number generators for use in cryptographic key generation are inadequate for machine learning, where some applications make heavy use of random inputs about which very specific assumptions are made (accepted for Usenix 2024)

Defacement Attacks on Israeli Websites is a measurement study of attacks by Palestinian sympathisers on Israeli websites since the Hamas attack on Israel (CW blog). Getting Bored of Cyberwar is a similar study of how pro-Ukrainian hackers responded to the Russian invasion of their country by attacking Russian websites, and pro-Russian hackers then responded (AP SC Magazine The Record)

No Easy Way Out: the Effectiveness of Deplatforming an Extremist Forum to Suppress Hate and Harassment is a measurement study of the industry attempt to take down Kiwi Farms in 2022-23. This holds a number of practical lessons for people interested in online censorship, as well as raising legal and philosophical issues with the approach taken by the UK's Online Safety Bill (The Register; accepted for Oakland 2024)

The Curse of Recursion: Training on Generated Data Makes Models Forget asks what will happen to GPT-{n} once most of the content online is generated by previous models. We show that the use of model-generated content in training leads to irreversible defects in subsquent model generations as the tails of the original distributions disappear, leading to model collapse (The Atlantic, Wall Street Journal, New Scientist, Venture Beat, Business Insider blog)

One Protocol to Rule Them All? On Securing Interoperable Messaging analyses the EU DMA mandate for messaging systems interoperability. This will vastly increase the attack surface at every level in the stack (blog Register Schneier).

Threat Models over Space and Time: A Case Study of E2EE Messaging Applications shows how Signal Desktop and WhatsApp Desktop are insecure; an opponent with temporary access to your laptop, such as a border guard or an intimate partner, can make this access persistent.

Chat Control or Child Protection debunks the arguments used by the intelligence community that "because children" we needed the Online Safety Bill which gave Ofcom the power to mandate snooping software in your phone (blog). The same arguments were used to support the so-called Child Sex Abuse Regulation which thankfully failed in the European Parliament (blog evidence video) – our big policy win of 2023.

Cambridge forced me to retire in September 2023 when I turned 67, a policy of unlawful age discrimination against which we are campaigning. I am now 20% at Edinburgh and (officially) 20% at Cambridge. I'm teaching a course in Security Engineering at Edinburgh to masters students and fourth-year undergrads, and the lecture videos are now all online (as are the lecture videos and notes for my first-year undergrad course on Software and Security Engineering at Cambridge).

timeline ...

The research students I advise are Bill Marino, Eleanor Clifford, Lawrence Piao, Jenny Blessing, Nicholas Boucher, Anh Viet Vu, and David Khachaturov. My RAs are Richard Clayton and Hridoy Dutta. I also work with Robert Brady. My former RAs are Sergei Skorobogatov, Lydia Wilson, Franck Courbon, Maria Bada, Yi Ting Chua, Ben Collier, Helen Oliver, Ildiko Pete, Daniel Thomas, Alice Hutchings, Sergio Pastrana, David Modic, Sven Übelacker, Julia Powles, Ramsey Faragher, Sophie van der Zee, Mike Bond, Vashek Matyas, Steven Murdoch, Andrei Serjantov and Alex Vetterl. My former students Jong-Hyeon Lee, Frank Stajano, Fabien Petitcolas, Harry Manifavas, Markus Kuhn, Ulrich Lang, Jeff Yan, Susan Pancho-Festin, Mike Bond, George Danezis, Sergei Skorobogatov, Hyun-Jin Choi, Richard Clayton, Jolyon Clulow, Hao Feng, Andy Ozment, Tyler Moore, Shishir Nagaraja, Robert Watson, Hyoungshick Kim, Shailendra Fuloria, Joe Bonneau, Wei-Ming Khoo, Rubin Xu, Laurent Simon, Kumar Sharad, Shehar Bano, Dongting Yu, Khaled Baqer, Alex Vetterl, Mansoor Ahmed and Ilia Shumailov have earned PhDs.

I'm teaching three Cambridge courses in 2023-24: the undergraduate course in Software and Security Engineering and graduate courses in Computer Security and Cybercrime. I also organise our security seminars and help run the Cambridge Cybercrime Centre.

My research topics include:

• Machine learning and signal processing – from adversarial machine learning to side channels
• Sustainability of security – from software patching through energy management to fighting wildlife crime
• Economics, psychology and criminology of information security – from dependability to deception detection
• Peer-to-Peer and social network systems – including the Eternity Service, cocaine auctions and suicide bombing
• Reliability of security systems – including bank fraud and hardware hacking
• Robustness of cryptographic protocols – including API attacks
• Cryptography – including why quantum crypto security proofs based on entanglement are convincing
• Security of clinical information systems – including ethics, genomic privacy and the care.data scandal
• Privacy and freedom issues – including chat control, "Keys under Doormats" and the Online Safety Act

The detection and manipulation of patterns, both overt and covert, has many applications, and the field is being refreshed by the recent revolution in neural networks.

• Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks AI Security shows that the randomness tests long used to check random number generators for use in cryptographic key generation are inadequate for machine learning, where some applications make heavy use of random inputs about which very specific assumptions are made.
• The Curse of Recursion: Training on Generated Data Makes Models Forget asks what will happen to GPT-{n} once most of the content online is generated by previous models. We show that the use of model-generated content in training leads to irreversible defects in subsquent model generations as the tails of the original distributions disappear, leading to model collapse (The Atlantic, Wall Street Journal, New Scientist, Venture Beat, Business Insider)
• Talking Trojan describes what we learned from trying to get industry to fix the Trojan Source vulnerability, which broke almost all computer languages, and the related Bad Characters vulnerability, which broke almost all NLP models. What parts of the disclosure ecosystem work, and which are broken? (blog) When Vision Fails then showed that the "obvious" defence to the Bad Characters attack, namely rendering text and then OCRing it, doesn't really work that well; and Boosting Big Brother: Attacking Search Engines with Encodings showed that the same techniques could be used for search engine optimisation and poisoning (blog).
• Trojan Source: Invisible Vulnerabilities shows how adversarial coding can make source code look different to a compiler and to a human reviewer. This enables supply-chain attacks to hide in plain sight (website blog).
• Bad Characters: Imperceptible NLP Attacks shows how the systems used for common natural-language processing tasks such as machine translation and toxic content filtering can be broken easily by inputs with adversarial coding. This can enable bad actors to hide in plain sight (website code).
• Markpainting: Adversarial Machine Learning meets Inpainting shows how to defeat inpainters – machine-learning tools that make it easy to edit or even forge images. Adversarial machine-learning tricks can be used to make images tamper-evident, or to add copyright marks that are extremely difficult for inpainters to remove (blog).
• Situational Awareness and Machine Learning – Robots, Manners and Stress argues that manners are a new frontier for research in robotics and machine learning. ML models find it really hard to interact with multiple humans, for example when an autonomous vehicle is trying to turn across traffic; this is related to situational awareness (blog).
• Data Ordering Attacks enable you to poison or backdoor a machine-learning system without changing the training data; you only have to manipulate the order in which the training samples are presented. For example, you can train a credit-scoring algorithm to be sexist by starting its training with ten rich men and ten poor women; but it's very much more general than that (blog).
• Sponge Examples: Energy-Latency Attacks on Neural Networks describes how to find inputs to neural networks that make them take a lot of time, or burn a lot of energy. They can be used to distract or to jam machine learning systems in a wide range of applications (blog press Schneier).
• Blackbox Attacks on Reinforcement Learning Agents Using Approximated Temporal Information demonstrates delayed-action attacks on reinforcement learning agents; some might be used as time bombs.
• Nudge Attacks on Point-Cloud DNNs disturb a small number of input points to a DNN to change how it classifies a 3-d object, and may therefore cause an autonomous vehicle or other robot to misunderstand its environment. We show two ways to generate them.
• The Taboo Trap is a mechanism we invented to block adversarial machine learning attacks on energy-constrained devices. An older version of paper was the subject of my invited talk at AISEC 2019. It emerged from earlier work on neural network compression, which appeared at SysML.
• The Taboo Trap work also led to further papers on transferability and adversarial reinforcement learning.
• Hey Alexa what did I just type? Decoding smartphone sounds with a voice assistant shows that if you type a password or PIN on a mobile phone within half a metre of a smart speaker with a good directional microphone array, the taps can give a lot of information about what you typed (New Scientist, Bruce Schneier, John Naughton, Daily Mail).
• BatNet: Data transmission between smartphones over ultrasound shows how to build a censorship-resistant mesh network using ultrasonic signals between smartphones. We also tested this as a covid contact-tracing technique; it turned out to be just as flaky as Bluetooth.
• Hearing your touch describes a new way to hack phones. A phone screen, like a drum, makes slightly different sounds depending on where you tap it, and given two microphones you can locate the tap too. So a hostile app can recover PIN codes and short words given a few measurements (blog Schneier).
• Don’t Interrupt Me While I Type: Inferring Text Entered Through Gesture Typing on Android Keyboards demonstrates a new side channel that can enable one Android app to steal another app's input (blog)
• In PIN Skimmer: Inferring PINs Through The Camera and Microphone we show that software on your smartphone can work out what PIN you enter on your phone by watching your face through the camera and listening for the clicks as you type (blog BBC CNN).
• Soft Tempest: Hidden Data Transmission Using Electromagnetic Emanations showed that the software on a computer can control its stray electromagnetic emanations. This can be used for both attack and defence. There's also a followup paper on the costs and benefits of Soft Tempest.
• Hollywood once hoped that copyright-marking systems would help control the copying of videos, music and computer games. This became high drama when a paper that showed how to break the DVD/SDMI copyright marking scheme was pulled by its authors from the Information Hiding 2001 workshop, following legal threats from Hollywood. In fact, the basic scheme – echo hiding – was among a number that we broke in 1997: see Attacks on Copyright Marking Systems. We also wrote Information Hiding – A Survey, which is a good place to start. There is much more on the web page of my former student Fabien Petitcolas.
• Another novel application of information hiding is the Steganographic File System. It will give you any file whose name and password you know, but if you do not know the correct password, you cannot even tell that a file of that name exists in the system! Its main function is to protect users against coercion. Two of our students implemented it: a paper is here, while the code is here. This functionality has since appeared in a number of crypto products.
• The threat in the 1990s by some governments to ban cryptography led to a surge of interest in steganography – the art of hiding messages in other messages – and then the surge of paranoia post-9/11 stoked interest in looking for them, with nonsense like this boosting many a bureaucrat's budget. Our paper On The Limits of Steganography explored what can and can't be done (here's an earlier version).
• The Newton Channel settles a conjecture of Simmons by exhibiting a high bandwidth subliminal channel in the ElGamal signature scheme.

Our computers and communications use several percent of global energy, and have secondary costs too – particularly if you have to throw things away for lack of software updates. I also have a long-standing interest in energy management and have more recently been looking at the energy wasted by cryptocurrency mining and at the prevention of wildlife crime. (Incidentally, this website is entirely static – no ads, trackers, javascript or even cookies. The estimated carbon cost per page view is 0.07g compared with over 2g for a typical commercial web page.)

• Making security sustainable is the new grand challenge for computer science: designing software so that durable goods such as cars can last longer (video of talk at 36C3 blog).
• Standardisation and Certification in the Internet of Things is an analysis of what happens to safety regulation once we get software everywhere. It informed EU directive 2019/771 which requires firms selling goods with digital components to maintain the software for at least two years, or for the reasonable expectation of the customer if longer. This will probably mean ten years for cars and white goods (blog).
• Privacy for Tigers describes work we did to stop wildlife aggregation sites being exploited by poachers.
• Bitcoin Redux examines what’s gone wrong in the world of cryptocurrencies, whose mining wastes colossal amounts of energy; financial regulators bear some of the blame for failing to enforce existing laws that would have prevented some of the worst abuses (blog). It follows on from Making Bitcoin Legal, where we presented a better way of tracing stolen bitcoin (blog video).
• What you get is what you C describes a compiler plugin we wrote to make it easier to maintain crypto code by expressing programmer intent.
• DigiTally is a prototype payment system we built to extend mobile phone payments to areas of less developed countries with no phone service.
• The UK smart meter project looks set to waste £20bn without saving any energy. Here are papers on the technical security and security economics of smart meters, on their privacy, and on their deployment.
• On the Reliability of Electronic Payment Systems describes work I did to help develop prepayment utility metering, which made possible the electrification of millions of homes in less developed countries. The STS standard we developed is now used in 400m meters in over 100 countries.

Incentives matter as much as technology for the security of large-scale systems. Systems break when the people who could fix them are not the people who suffer the costs of failure. So it's not enough for security engineers to understand cryptomathematics and the theory of operating systems; we have to understand game theory and microeconomics too. I pioneered the discipline of security economics which is starting to embrace privacy economics, security psychology and criminology too.

• No Easy Way Out: the Effectiveness of Deplatforming an Extremist Forum to Suppress Hate and Harassment is a measurement study of the industry attempt to take down Kiwi Farms in 2022-23. This holds a number of practical lessons for people interested in online censorship, as well as raising legal and philosophical issues with the approach taken by the UK's Online Safety Bill (The Register)
• Defacement Attacks on Israeli Websites is a measurement study of attacks by Palestinian sympathisers on Israeli websites since the Hamas attack on Israel (CW blog)
• Getting Bored of Cyberwar: Exploring the Role of Civilian Hacktivists in the Russia-Ukraine Conflict is a measurement study of how pro-Ukrainian hackers responded to the Russian invasion of their country by attacking Russian websites, and pro-Russian hackers then responded (AP SC Magazine The Record)
• ExtremeBB is a database we have collected of more than 50m posts to underground extremist forums, and which we make available to social scientists studying violent online political extremism, misogyny, radicalisation and hate speech. This exercise taught us about the strong correlation between misogyny and terrorist violence, whether from Islamists or the far right (blog)
• PostCog is a search engine we're building to make access and analysis easier for users of ExtremeBB.
• Silicon Den: Cybercrime is Entrepreneurship analyses underground criminal enterprises as tech startups; their main impediment compared with regular tech businesses may be lack of access to finance (blog)
• The gift of the gab: Are rental scammer skilled at the art of persuasion? studies accommodation frauds perpetrated against Cambridge students and postdocs. The fraudsters use standard boilerhouse sales techniques, and succeed because law enforcement ignore them (blog).
• Bitcoin Redux explains what’s gone wrong in the world of cryptocurrencies (blog). It follows on from Making Bitcoin Legal, which describes a better way of tracing stolen bitcoin (blog video).
• Taking Down Websites to Prevent Crime analyses the takedown industry. Private firms are better at taking down websites than the police; they do a lot more of it!
• Reconciling Multiple Objectives –- Politics or Markets? discusses how institutional economics can help explain how protocols evolve (blog).
• When Lying Feels the Right Thing to Do reports that people are more likely to lie when they feel rejected (blog blog press).
• It’s All Over but the Crying: The Emotional and Financial Impact of Internet Fraud shows that fraud victims suffer significant emotional harm as well as financial loss (blog followup)
• To freeze or not to freeze shows how you may be able to build a better lie detector by analysing body motion, while Mining Bodily Cues to Deception, analyses the signals that can be extracted from different limb movements (blog Guardian Mail).
• Experimental Measurement of Attitudes Regarding Cybercrime discusses how prosecutors and public opinion are out of step; the former consider protest crimes to be more serious than crimes done for financial gain, while voters take the opposite view.
• We will make you like our research: the development of a susceptibility-to-persuasion scale presents a questionnaire for determining how gullible fraud victims are, and indeed how vulnerable people are in general to manipulation by marketers (SSRN blog).
• Reading this may harm your computer – The psychology of malware warnings analyses what sort of text we should put in a warning if we actually want the user to pay attention to it (blog).
• Measuring the Cost of Cybercrime sets out to debunk the scaremongering around online crime that governments and defence contractors are using to justify everything from increased surveillance to preparations for cyberwar. It was written in response to a request from the UK Ministry of Defence, and appeared at WEIS 2012 (press: BBC PC World Computerworld)
• We wrote a major report for ENISA on the Resilience of the Internet interconnection ecosystem which has been adopted as ENISA policy. We believe this is the first time anyone has documented how the Internet actually works in practice, as opposed to in theory. This link will take you to both the full report (238 pages) and the 31-page executive summary.
• Tyler Moore and I wrote a series of survey papers on security economics as research in the field got going. The 2011 tech report Economics and Internet Security: a Survey of Recent Analytical, Empirical and Behavioral Research appeared later as a book chapter. An earlier survey, Information Security Economics – and Beyond, appeared in various versions from 2006 to 2009. There was a short survey in Science in late 2006; a version for economists at Softint in January 2007; a version for security engineers at Crypto in August 2007 (see slides); a book chapter for mathematicians; and finally an archival journal version in Phil Trans Roy Soc A (Aug 2009).
• Security Economics – A Personal Perspective is an invited talk I gave at ACSAC 2012 telling the history of security economics slides).
• It's the Anthropology, Stupid! discusses how we might put context and emotion back into security decisions.
• The Economics of Online Crime appeared in the Journal of Economic Perspectives; it looks at the econometrics of fraud and phishing, and makes a number of suggestions for improving the responses of banks and law-enforcement agencies.
• The Impact of Incentives on Notice and Take-down examines how take-down speed varies as a function of incentives. Banks are quick to remove phishing websites that mention them by name, but they ignore mule recruitment websites because it's harder to tell which bank will be affected.
• We have two futher papers on security economics in banking. The first is on Verified by VISA – the mechanism that asks for your card password when you shop online. This is an example of how a poor design can win out if it has strong deployment incentives (see also blog post and slides). The second, On the Security of Internet Banking in South Korea, analyses the effects of Korea's decision to use national cryptography standards for Internet banking rather than just using the same protocols as the rest of the world.
• On the security economics of electricity metering appeared at WEIS 2010 and warns that the government's smart meter programme probably won't work. Other papers on security economics and control systems include Security Economics and Critical National Infrastructure (at WEIS 2009); Certification and Evaluation (at IEEE ETFA 2009); The Protection of Substation Communications (SCADA Security Scientific Symposium, 2010); and Towards a security architecture for substations (IEEE PES – ISGT Europe, 2011).
• The Trust Economy of Brief Encounters argues that as transactions become more transient, we will have to authenticate more; it appeared at the protocols workshop in 2009.
• We did a major study of Security Economics in the Single Market for the European Network and Information Security Agency. We looked at the market failures underlying spam, phishing and other online problems, and made concrete policy proposals, some of which have been adopted. A shorter version (62 pages) appeared at WEIS 2008 (slides) and an even shorter version (25 pages), at ISSE 2008.
• Closing the Phishing Hole – Fraud, Risk and Nonbanks reports research on payment regulation commissioned by the US Federal Reserve. This paper identified speedy asset recovery as the best way to deter online fraud and rapid, irrevocable payment instruments (such as Western Union) as a systemic threat.
• Why Information Security is Hard – An Economic Perspective was the paper that got information security people thinking about economics. It applies microeconomic analysis to explain many phenomena that security folks had found to be pervasive but perplexing.
• My `Trusted Computing' FAQ undermined the Trusted Computing Group's initiative to install DRM hardware in every computer, PDA and mobile phone. `TC' was sold to Hollywood as a DRM platform but its real beneficiary would have been the software industry whose customers would have been locked in more tightly. Cryptography and Competition Policy – Issues with `Trusted Computing' is an economic analysis I gave at WEIS2003 and as an invited talk at PODC 2003. A short version of the paper appeared in Cepis Upgrade). I spoke about TC at the "Trusted Computing Group" Symposium, which helped drive German and EU policy. The row was ignited by a paper on the security of free and open source software I gave at Softint 2002; see coverage in the New York Times and The Register.
• In my paper on the security of free and open source software, I show that the old argument whether source code access makes it easier for the defenders to find and fix bugs, or easier for the attackers to find and exploit them is misdirected. Under standard assumptions used in reliability growth modelling, the two will exactly cancel each other out. That means that whether open or closed systems are more secure in a given situation will depend on whether, and how, the application deviates from the standard assumptions. These ideas are developed further in Open and Closed Systems are Equivalent (that is, in an ideal world) which appeared as a chapter in Perspectives on Free and Open Source Software. See press coverage in slashdot, news.com and The Register.
• On Dealing with Adversaries Fairly applies election theory (also known as social choice theory) to the problem of shared control in distributed systems.
• The Economics of Censorship Resistance examines when it is better for defenders to aggregate or disperse. Should file-sharers build one huge system like gnutella and hope for safety in numbers, or should everyone just share the stuff they care about? More generally, what are the tradeoffs between diversity and solidarity when conflict threatens? (This was starting to be a live topic in social policy, and has led to a lot of research since.) Our paper appeared at WEIS 2004.
• Here are papers on The Initial Costs and Maintenance Costs of Protocols, which appeared at Security Protocols 2005, and How Much is Location Privacy Worth? from WEIS 2005.

There are two relevant workshops I helped establish: Security and Human Behaviour workshop which brings together security engineers and psychologists, while the Workshop on Economics and Information Security is where you meet everyone working in security economics. hundred participants.

One of the seminal papers in peer-to-peer systems was The Eternity Service, which I invented in response to growing Internet censorship, The modern era only started once the printing press enabled seditious thoughts to be spread too widely to ban. But when books no longer exist as tens of thousands of paper copies, but as a file on a single server, will courts be able to order them unpublished once more? (This has since happpened to newspaper archives in Britain.) So I invented the Eternity Service as a means of putting electronic documents beyond the censor's grasp. It inspired second-generation censorship-resistant systems such as Publius and Freenet; one descendant is wikileaks. But the killer app turned out to be not sedition, or even pornography, but copyright. Hollywood's action against Napster led to our ideas being adopted in filesharing systems; they are now re-emerging in the Internet of Things.

Work since the Eternity paper includes the following.

• Do You Believe in Tinker Bell? The Social Externalities of Trust explores how we can crowdsource trust. Just as a religion's power comes from its faithful rather than from the government, so also a trust service's power should derive from the users who trust it, rather than from a CA that's too big to fail (blog)
• An Experimental Evaluation of Robustness of Networks studies the best attack and defence strategies in different kinds of networks. It builds on an earlier paper, the topology of covert conflict, which asked how the police can best target an underground organisation given some knowledge of its patterns of communication, and how might they in turn might react to various law-enforcement strategies.
• Social Authentication – harder than it looks shows how Facebook's social captcha system is vulnerable to guessing by friends and to face recognition software (blog)
• Temporal Node Centrality in Complex Networks proposes new metrics for analysing highly dynamic systems. If there's an epidemic of flu, should you close down the schools or the subway? (blog news)
• Centrality Prediction in Dynamic Human Contact Networks examines empirical methods for predicting centrality of individuals in different contact networks that evolve over time.
• Eight Friends are Enough: Social Graph Approximation via Public Listings shows how easy it is for an outsider to work out the structure of friendships on Facebook (see our blog on Facebook's technical privacy and its democracy theatre.)
• New Strategies for Revocation in Ad-Hoc Networks analyses when it makes economic sense to use suicide bombing as a tactic. Suicide attacks are found widely in nature, from bees to helper T-cells; this model may help explain why (press coverage here and here). The idea was developed further in Fast exclusion of errant devices from vehicular networks.
• The Resurrecting Duckling: Security Issues for Ad-hoc Wireless Networks describes how to do key management between low-cost devices without either the costs or privacy problems of central servers: trust on first use, followed by a factory reset if need be. (There's also a journal version here.)
• The trust-on-first-use technique was first used at scale in digital tachographs.
• Key Infection – Smart trust for Smart Dust applied the approach to ad-hoc networks. Peers establish keys opportunistically, and you work out how to recover from later node compromise.
• Homeplug AV is an industry standard I helped design for broadband communication over the power mains, which is widely used in wireless LAN extenders. It also uses trust-on-first-use key management, and the critical problem turned out to be: how do you recover if you don't recruit the right device, but a similar one nearby?
• Sybil-resistant DHT routing shows how we can make peer-to-peer systems more robust against disruptive attacks if we know which nodes introduced which other nodes.
• The Economics of Censorship Resistance examines when it is better for defenders to aggregate or disperse. Should file-sharers build one huge system like gnutella and hope for safety in numbers, or would a loose federation of fan clubs for different bands work better?
• A keynote talk about next-generation peer-to-peer systems at Wizards of OS 2004 discussed how Usenet might be reimplemented.
• A New Family of Authentication Protocols presented our "Guy Fawkes Protocol", which lets users sign messages using only two computations of a hash function and one reference to a timestamping service. It led to the Tesla research on protocols for signing digital streams.
• The Cocaine Auction Protocol explored how transactions can be conducted between mutually mistrustful principals with no trusted arbitrator, while giving a high degree of privacy against traffic analysis.
• The Eternal Resource Locator: An Alternative Means of Establishing Trust on the World Wide Web investigated how to embed trust mechanisms in html documents. It grew out of a medical school project to protect drug data; for details, see Secure Books: Protecting the Distribution of Knowledge. We also looked at how to secure a digital repository. This evolved into Jikzi, an authentication service which also caches links on which you've relied.
• The XenoService – A Distributed Defeat for Distributed Denial of Service described defeating DDoS attacks using a network of web hosts that can respond to an attack on a site by replicating it rapidly and widely. It used Xen, a hypervisor developed at Cambridge for distributed hosting, which led to another startup.

• Attack of the Clones: Measuring the Maintainability, Originality and Security of Bitcoin 'Forks' in the Wild shows how most altcoins are scams and compares various ways of spotting them, such as the lack of novel code, the lack of bug fixes, and the lack of maintenance generally.
• Making Security Sustainable discusses a new challenge: how we will manage to continue patching cars and other safety-critical durable goods for decades (blog)
• What you get is what you C: Controlling side effects in mainstream C compilers shows how our toolsmiths could be our allies rather than a subversive fifth column in our rear (blog).
• Standardisation and Certification in the Internet of Things discusses what happens when we get software everywhere. Security will be more and more about safety. There will be many fasincating engineering challenges. The paper is a short version of a big report we did for the Europen Commission on the future of safety regulation – which will need a serious rethink! (short video longer video Prospect blog)
• International Comparison of Bank Fraud Reimbursement is a comparative study of the security advice banks give their customers, of whether customers understand it, and whether they think it fair (blog press)
• Our Security Analysis of Factory Resets shows that in most Android phones, the factory reset function doesn't work very well; it's usually possible to recover credentials for gmail and other services along with personal data. Our Security Analysis of Consumer-Grade Anti-Theft Solutions Provided by Android Mobile Anti-Virus Apps shows that third-party security offerings are no better (blog The Verge Ars Technica Register BBC).
• Collaborating with the enemy on network management describes a project to develop a version of Quagga for software defined networking research. It appeared at Security Protocols 2014. Authentication for Resilience: the Case of SDN discusses the authentication problems we need to solve, and appeared at Security Protocols 2013.
• Rendezvous is a prototype search engine for code, which recasts decompilation as a search problem (blog).
• Be Prepared: The EMV Pre-play Attack discloses a family of vulnerabilities in EMV, the protocol underlying Chip and PIN payments. This may explain many disputed transactions that look like card cloning and which the banks often refuse to refund to fraud victims (blog BBC FT PCW Schneier) conference version of paper )
• How Certification Systems Fail: Lessons from the Ware Report analyses failures in the Common Criteria, FIPS 140 and other certification mechanisms by studying lessons from the banking industry.
• CHERI: a research platform deconflating hardware virtualization and protection is the first paper on a large project we have with SRI to build a CPU supporting capabilities, port FreeBSD to it, and build some demonstrator apps exploring the costs and benefits of CPUs with hardware support for more fine-grained access control.
• Aurasium: Practical Policy Enforcement for Android Applications describes how to repackage Android apps to add user sandboxing and policy-enforcement code (source code).
• Risk and privacy implications of consumer payment innovation discusses what threats to competition, privacy and payment security might arise as a result of mobile innovation; I gave it at the Fed's biennial Payment Systems Conference (slides blog).
• A birthday present every eleven wallets? is the first proper study of the security of customer-selected bank PINs, and documents all sorts of bad stuff (blog, press, blog).
• Who controls the off switch? describes the strategic vulnerability created by the UK plan to replace 47m gas and electricity meters with ‘smart meters’ that can be switched off remotely. There are further papers on SCADA security engineering here, here, here, here and here.
• A recurrring theme is the vulnerabilities in the EMV payment system, known in the UK as Chip and PIN. We won an award for a paper describing a man-in-the-middle attack that allows a stolen card to be used with any pin. There was a TV piece on Newsnight; see also ZDnet, the Telegraph, the Mail, the Mirror, the Register, Bruce Schneier, the press release and our FAQ. Rather than fixing the problem, the UK banks sought to suppress our research; see comment in the Guardian, the Indy, the Mail, the Cambridge News, Slashdot, Ars Technica, Radio 4 and Radio 5).
• Other recent work on problems with bank systems includes Can We Fix the Security Economics of Federated Authentication? which explores how we can deal with a world in which your mobile phone contains your credit cards, your driving license and even your car key (blog); a paper on whether EMV is bad for innovation; a paper on Verified by VISA, the mechanism that asks for your card password when you shop online; and a tech report On the Security of Internet Banking in South Korea.
• Optimised to Fail: Card Readers for Online Banking documents the shortcomings of the CAP card readers used for online banking; see also our blog, press coverage and the later journal version.
• Thinking inside the box: system-level failures of tamper proofing documented serious vulnerabilities in Chip and PIN payment terminals and won the Best Practical Paper award at the 2008 Oakland conference. It was also featured on Newsnight. Here are some frequently asked questions, our press release, and coverage in the Register, the Newsnight blog and the Telegraph. My paper Failures on Fraud appeared in a central bankers' magazine and argued that all this is yet another symptom of the failure of bank regulation.
• The snooping dragon: social-malware surveillance of the Tibetan movement explains how the Chinese intelligence services compromised many of the computers at the Dаlai Lаma's private office, and what this means for information security (also slides).
• Why Cryptosystems Fail was my first widely-cited paper and the first on what goes wrong with payment systems. This version appeared at ACMCCS 93 and explains how ATM fraud was done in the early 1990s. Liability and Computer Security – Nine Principles took this work further, and examines the problems with relying on cryptographic evidence. The recent introduction of EMV ('chip and PIN') was supposed to fix the problem, but hasn't: Phish and Chips documents protocol weaknesses in EMV, and A Note on EMV Secure Messaging in the IBM 4758 CCA documents even more. The Man-in-the-Middle Defence shows how to turn protocol weaknesses to advantage. See my paper RFID and the Middleman for the likely next wave of frauds.
• On a New Way to Read Data from Memory describes techniques we developed that use lasers to read out memory contents directly from a chip, without using the read-out circuits provided by the vendor. The work builds on methods described in Optical Fault Induction Attacks, which showed how laser pulses could be used to induce faults in smartcards that would leak secret information. That paper appeared at CHES 2002; it made the front page of the New York Times and also got covered by slashdot. It led to the field of semi-invasive attacks on semiconductors, pioneered by my then research student Sergei Skorobogatov.
• After we discovered the above attacks, we developed a CPU technology that uses redundant failure-evident logic to thwart attacks based on fault induction or power analysis. Our first paper on this technology won an award at Async 2002. Our journal paper, Balanced Self-Checking Asynchronous Logic for Smart Card Applications, has more.
• Our classic paper on hardware security, Tamper Resistance – A Cautionary Note, describes how to penetrate the smartcards and secure microcontrollers of the mid-1990s. It kicked off the modern academic study of hardware security and won a Best Paper award. Our second paper on the subject was Low Cost Attacks on Tamper Resistant Devices, which describes a number of further tricks. See also the home page of our hardware security laboratory, and Markus Kuhn's page of links to hardware attack resources.
• On the Reliability of Electronic Payment Systems describes work I did to help develop prepayment utility metering, which made possible the electrification of millions of homes in Africa. It appeared in the May 1996 issue of the IEEE Transactions on Software Engineering. An ealier version, entitled Cryptographic Credit Control in Pre-Payment Metering Systems, appeared at Oakland 95. A later paper on this subject discussed how we could apply what we'd learned to support utility meter interworking in the UK after deregulation.
• On the Security of Digital Tachographs successfully predicted how the introduction of smartcard-based digital tachographs throughout Europe from 2005 would affect fraud and tampering.
• How to Cheat at the Lottery reports a novel and, I hope, entertaining experiment in software requirements engineering.
• The Grenade Timer describes a novel way to protect low-cost processors against denial-of-service attacks, by limiting the number of cycles an application can consume.
• The Millennium Bug – Reasons Not to Panic describes our experience in coping with the bug at Cambridge University and elsewhere. This paper correctly predicted that the bug wouldn't bite very hard. Journalists were not interested. I later discussed what we could learn from the incident in a radio interview with Stephen Fry.
• The Memorability and Security of Passwords – Some Empirical Results tackles an old problem – how do you train users to choose passwords that are easy to remember but hard to guess? We did a randomized controlled trial with a few hundred first year science students which confirmed some folk beliefs, but debunked some others. This became one of the classic papers on security usability.
• Murphy's law, the fitness of evolving species, and the limits of software reliability applies the techniques of statistical thermodynamics to the failure modes of any complex system that evolves under testing. The resulting reliability growth model is in close agreement with empirical data, and inspired later work in security economics.
• Security Policies play a central role in secure systems engineering. They provide a concise statement of the kind of protection a system is supposed to achieve. This article is a security policy tutorial.
• Combining cryptography with biometrics shows that in those applications where you can benefit from biometrics, you often don't need a large central database (as proposed for Britain's ID card). There are smarter and less privacy-invasive ways to arrange things.

The papers on physical security by Roger Johnston's team are also definitely worth a look; see also an old leaked copy of the NSA Security Manual.

• One Protocol to Rule Them All? On Securing Interoperable Messaging analyses the EU DMA mandate for messaging systems interoperability. This will vastly increase the attack surface at every level in the stack – from the cryptography up through usability to commercial incentives and the opportunities for government interference. It will be complexity on steroids (blog Register Schneier).
• Threat Models over Space and Time: A Case Study of E2EE Messaging Applications shows how Signal Desktop and WhatsApp Desktop are insecure; an opponent with temporary access to you laptop, such as a border guard or an itimate partner, can make this access persistent.
• CoverDrop: Securing Initial Contact for Whistleblowers is a better way for a newspaper to help anonymous sources get in touch. By hiding traffic in the paper's own app, it prevents the traffic-analysis attacks that are possible against users of SecureDrop and Signal (blog.
• DigiTally: Piloting Offline Payments for Phones reports a field trial of a system we designed to extend mobile phone payments to places with no phone service. The protocol design itself is described in SMAPs: Short Message Authentication Protocols (blog slides discussion).
• API Level Attacks on Embedded Systems are a powerful way to attack cryptographic processors, and indeed any systems where more trusted processes talk to less trusted ones. We found that a "secure" device can often be defeated by sending it some sequence of transactions which its designer did not expect. We've defeated pretty well every security processor we've looked at, at least once. This line of research started at Protocols 2000 with The Correctness of Crypto Transaction Sets; more followed in the first edition of my book. Robbing the bank with a theorem prover shows how to apply advanced tools to the problem, and ideas for future research can be found in Protocol Analysis, Composability and Computation. For a snapshot of how this interacts with physical security, see our survey of cryptographic processors, a shortened version of which appeared in the February 2006 Proceedings of the IEEE. An up-to-date survey of API attacks can be found in the second edition of my my book. There is also an API security FAQ and an annual workshop.
• Security protocols and evidence: where many payment systems fail analyses why dispute resolution is hard. In a nutshell, the systems needed to support it properly just don't get built (blog).
• Authentication for Resilience: the Case of SDN discusses the authentication problems we need to solve if we're to move software defined networks out of the data centre into more heterogeneous environments. It appeared at Security Protocols 2013.
• Can We Fix the Security Economics of Federated Authentication? explores how protocols work, or fail, at global scale. How can we deal with a world in which your mobile phone contains your credit cards, your driving license and even your car key – and in particular what happens when it gets stolen or infected? (blog)
• Key Management for Substations: Symmetric Keys, Public Keys or No Keys? debunks the proposal to mandate public-key crypto in electricity substations. In this particular application, the right solution is usually to have no crypto at all.
• What Next after Anonymity? argues that it isn't enough to worry about the confidentiality of metadata (anonymity); we sometimes need to protect their integrity as well.
• Programming Satan's Computer is a phrase Roger Needham and I coined to express the difficulty of designing cryptographic protocols; it has recently been popularised by Bruce Schneier (see, for example, his foreword to my book). The problem of designing programs which run robustly on a network containing a malicious adversary is rather like trying to program a computer which gives subtly wrong answers at the worst possible moment.
• Robustness principles for public key protocols gives a number of attacks on protocols based on public key primitives. It also puts forward some principles which can help us to design robust protocols, and to find attacks on other people's designs. It appeared at Crypto 95.
• The Cocaine Auction Protocol explores how transactions can be conducted between mutually mistrustful principals with no trusted arbitrator, even in environments where anonymous communications make most of the principals untraceable.
• The Initial Costs and Maintenance Costs of Protocols appeared at the 2005 Protocols Workshop and shows how economics can enter into protocol design.
• NetCard - A Practical Electronic Cash Scheme presents research on micropayment protocols for use in electronic commerce. We invented tick payments simultaneously with Torben Pedersen and with Ron Rivest and Adi Shamir; we all presented our work at Protocols 96.
• The GCHQ Protocol and its Problems pointed out a number of flaws in a key management protocol promoted by GCHQ as a European alternative to Clipper, until we shot it down with this paper at Eurocrypt 97. Many of the criticisms we developed here also apply to the more recent, pairing-based cryptosystems.
• The Formal Verification of a Payment System describes the first use of formal methods to verify an actual payment protocol, which was (and still is) used in an electronic purse product (VISA's COPAC card). This is a teaching example I use to get the ideas of the BAN logic across to undergraduates. There is further detailed information in a technical report, which combines papers given at ESORICS 92 and Cardis 94.
• An Attack on Server Assisted Authentication Protocols appeared in Electronics Letters in 1992. It breaks a digital signature protocol.
• On Fortifying Key Negotiation Schemes with Poorly Chosen Passwords presents a simple way of achieving the same result as protocols such as EKE, namely preventing middleperson attacks on Diffie-Hellman key exchange between two people whose shared secret could be guessed by the enemy.

Protocols have been the stuff of high drama. Citibank asked the High Court to gag the disclosure of certain crypto API vulnerabilities that affect a number of systems used in banking. I wrote to the judge opposing this; a gagging order was still imposed, although in slightly less severe terms than Citibank had requested. The trial was in camera, the banks' witnesses didn't have to answer questions about vulnerabilities, and new information revealed about these vulnerabilities in the course of the trial may not be disclosed in England or Wales. Information already in the public domain was unaffected. The vulnerabilities were discovered by Mike Bond and me while acting as the defence experts in a phantom withdrawal court case, and independently discovered by the other side's expert, Jolyon Clulow, who later joined us as a research student. They are of significant scientific interest, as well as being relevant to the rights of the growing number of people who suffer phantom withdrawals from their bank accounts worldwide. Undermining the fairness of trials and forbidding discussion of vulnerabilities isn't the way forward (press coverage by the Register).

• Maxwell's fluid model of magnetism shows that a wavepacket travelling along a phase vortex in an Eulerian fluid obeys Maxwell's equations, is emitted and absorbed discretely, and can have linear or circular polarisation. What's more, the measured correlation between the polarisation of two cogenerated wavepackets is exactly the same as predicted by quantum mechanics, and observed in the Bell tests (blog press).
• If you're new to this subject, a good starting point is to watch the video of Yves Couder's beautiful bouncing-droplet experiments, and then read our paper Why bouncing droplets are a pretty good model of quantum mechanics. This shows how droplets bouncing on a vibrating fluid bath obey two-dimensional analogues of Maxwell's equations and a version of Schrödinger's equation.
• For the hard math, which explains how fermionic quasiparticles obeying Dirac's equation can arise in a bosonic fluid, see this paper; another paper that may be relevant is here. And here's a video of my talk at the 2015 Crossing conference, and another video on the various ways in which provable security fails (including the quantum case).

In the 1990s I worked with Eli Biham and Lars Knudsen to develop Serpent – a candidate block cipher for the Advanced Encryption Standard. Serpent got the second largest number of votes.

Other papers on cryptography and cryptanalysis include the following.

• The Dancing Bear – A New Way of Composing Ciphers presents a new way to combine crypto primitives. Previously, to decrypt using (say) any three out of five keys, the keys all had to be of the same type (such as RSA keys). With my new construction, you can mix and match - RSA, AES, even one-time pad. The paper appeared at the 2004 Protocols Workshop; an earlier version came out at the FSE 2004 rump session.
• Two Remarks on Public Key Cryptology is a note on two ideas I floated at talks I gave in 1997-98, concerning forward-secure signatures and compatible weak keys. The first of these has inspired later research by others; the second gives a new attack on public key encryption.
• Two Practical and Provably Secure Block Ciphers: BEAR and LION shows how to construct a block cipher from a stream cipher and a hash function. We had already known how to construct stream ciphers and hash functions from block ciphers, and hash functions from stream ciphers; so this paper completed the set of elementary reductions. It also led to the "Dancing Bear" above.
• Tiger – A Fast New Hash Function defines a new hash function, which we designed following Hans Dobbertin's attack on MD4. This was designed to run extremely fast on the new 64-bit processors such as DEC Alpha and IA64, while still running reasonably quickly on existing hardware such as Intel 80486 and Pentium (the above link is to the Tiger home page, maintained in Haifa by Eli Biham; if the network is slow, see my UK mirrors of the Tiger paper, new and old reference implementations (the change fixes a padding bug) and S-box generation documents. There are also third-party crypto toolkits supporting Tiger, such as that from Bouncy Castle).
• Minding your p's and q's points out a number of things that can go wrong with the choice of modulus and generator in public key systems based on discrete log. It elucidated some of the previously classified reasoning behind the design of the US Digital Signature Algorithm, and appeared at Asiacrypt 96.
• Chameleon – A New Kind of Stream Cipher shows how to do traitor tracing using symmetric rather than public-key cryptology. The idea is to turn a stream cipher into one with reduced key diffusion, but without compromising security. A single broadcast ciphertext is decrypted to slightly different plaintexts by users with slightly different keys. This paper appeared at Fast Software Encryption in Haifa in January 1997.
• Searching for the Optimum Correlation Attack shows that nonlinear combining functions used in nonlinear filter generators can react with shifted copies of themselves in a way that opens up a new and powerful attack on many cipher systems. It appeared at the second workshop on fast software encryption.
• The Classification of Hash Functions showed that correlation freedom is strictly stronger than collision freedom, and shows that there are many pseudorandomness properties other than collision freedom which hash functions may need. It appeared at Cryptography and Coding 93.
• A Faster Attack on Certain Stream Ciphers shows how to break the multiplex shift register generator, which is used in satellite TV systems. I found a simple divide-and-conquer attack on this system in the mid 1980's, a discovery that got me "hooked" on cryptology. This paper is a refinement of that work.
• On Fibonacci Keystream Generators appeared at FSE3, and shows how to break "FISH", a stream cipher proposed by Siemens. It also proposes an improved cipher, "PIKE", based on the same general mechanisms.
• Tree Functions and Cipher Systems appeared in 1991; it points out a weakness in a proprietary cipher that was later developed into this.

Another of my contributions was founding the series of workshops on Fast Software Encryption.

The NHS has a long history of privacy abuses. Gordon Brown's own medical records were compromised while he was prime minister, but the miscreant got off scot-free as it was "not in the public interest" to prosecute him. In another famous case, Helen Wilkinson had to organise a debate in Parliament to get ministers to agree to remove defamatory and untrue information about her from NHS computers. The minister assured the House that the libels had been removed; months later, they still had not been. There is now an NGO set up specifically to campaign for health privacy, medConfidential.org.

Here are my most recent papers on the subject.

• Confidentiality in Remote Clinical Practice is a report I wrote for the International Psychoanalytical Association, analysing what we learned before and during the pandemic on the safety and privacy of remote psychotherapy.
• The collection, linking and use of data in biomedical research and health care: ethical issues is a report we wrote for the Nuffield Bioethics Foundation: what happens to health privacy in a world with cloud-based medical records and pervasive genomics? (blog Guardian Indy Press Association Science)
• Database State is a report we wrote for the Joseph Rowntree Reform Trust on the failings of public-sector IT in Britain, and how to fix them. It pointed out that a number of health systems almost certainly break European law. There's coverage on the BBC, in the Guardian (also here), the Mail (also here), the Independent, the Telegraph, E-Health Insider and Liberty Central. This report had a lot of impact; the coalition government promised to abolish or at least change a number of the systems we fingered as unlawful. Both the Conservatives and the Lib Dems promised to axe the NHS centralisation project too if they won the 2010 election; after they did so, the name was changed but the stupidity continued.
• In 2006 I organised 23 computer science professors to write to the Health Committee requesting an independent review of the NHS National Programme for IT, the last big centralisation drive, as it was visible failing. Ministers refused, and NPfIT went on to become the largest civil-government IT project failure ever.
• I was one of the authors of a 2006 report on the safety and privacy of children's databases, done for the UK Information Commissioner. It concluded that government plans to link up most of the public-sector databases that hold information on children were misguided: the proposed systems would be both unsafe and illegal. This report got a lot of publicity. I spoke on these issues on these videos made by Action on Rights for Children.
• I wrote a report for the National Audit Office on the health IT expenditure, strategies and goals of the UK and a number of other developed countries. This showed that the NHS National Program for IT is in many ways an outlier, and high-risk.
• Here is an article I wrote for Drugs and Alcohol Today analysing the likely effects of the NHS computing project on patient privacy, particularly in the rehabilitation field.
• In 2007 I acted as a Special Adviser to the House of Commons Health Select Committee's Report on the Electronic Patient Record. (See also the parliamentary debate on the report, press comment, and an article on the implications for HIV treatment.)
• Patient confidentiality and central databases appeared in the February 2008 British Journal of General Practice, calling on GPs to encourage patients to opt out of the NHS care records service.
• System security for cyborgs discusses technical, ethical and security-economics issues to do with implantable medical devices.

Civil servants started pushing for online access to everyone's records in 1992 and I got involved in 1995, when I started consulting for the British Medical Association on the safety and privacy of clinical information systems. Back then, the police were given access to all drug prescriptions, after the government argued that they needed it to catch doctors who misprescribed heroin. The police got their data, but they didn't catch Harold Shipman, and no-one was held accountable. The NHS slogan in 1995 was `a unified electronic patient record, accessible to all in the NHS'. The BMA campaigned against this, arguing that it would destroy patient privacy:

• Security in Clinical Information Systems was published by the BMA in January 1996. It sets out rules that uphold the principle of patient consent independently of the details of specific systems. It was the medical profession's initial response to the safety and privacy problems posed by centralised NHS computer systems.
• An Update on the BMA Security Policy appeared in June 1996 and tells the story of the struggle between the BMA and the government, including the origins and development of the BMA security policy and guidelines.
• There are comments made at NISSC 98 on the healthcare protection profiles being developed by NIST for the DHHS to use in regulating health information systems privacy, which made a number of mistaken assumptions about threats and protection mechanisms.
• Remarks on the Caldicott Report raises a number of issues about the report of the Caldicott Committee, which was set up by the Major government to kick the medical privacy issue into touch until after the 1997 election. Its members failed to understand that medical records from which the names have been removed, but where NHS numbers remain, are not anonymous – as large numbers of NHS staff need to map names to numbers in order to do their jobs.
• Information technology in medical practice: safety and privacy lessons from the United Kingdom provided an overview of the safety and privacy problems we encountered in UK healthcare computing in the mid-90s for readers of the Australian Medical Journal.
• The DeCODE Proposal for an Icelandic Health Database analyses a proposal to collect all Icelanders' medical records into a single database. I evaluated this for the Icelandic Medical Association and concluded that the proposed security wouldn't work. The company running it soon hit financial problems and later filed for bankruptcy. The ethical issues were a factor: Iceland's Supreme Court allowed a woman to block access to her father's records because of the information they may reveal about her (see analysis). This effectively killed the vision of having the whole population on a database. I also wrote an analysis of security targets prepared under the Common Criteria for the evaluation of this database. See also BMJ correspondence and an article by Einar Arnason.
• Clinical System Security – Interim Guidelines appeared in the British Medical Journal on 13th January 1996. It advises healthcare professionals on prudent security measures for clinical data. The most common threat is that private investigators use false-pretext telephone calls to elicit personal health information from assistant staff.
• A Security Policy Model for Clinical Information Systems appeared at the 1996 IEEE Symposium on Security and Privacy. It presents the BMA policy model to the computer security community in a format comparable to policies such as Bell-LaPadula and Clark-Wilson. It had some influence on later US health privacy legislation (the Kennedy-Kassebaum Bill, now HIPAA).
• NHS Wide Networking and Patient Confidentiality appeared in the British Medical Journal in July 1995 and set out some early objections to the government's health network proposals.
• Patient Confidentiality &ndash At Risk from NHS Wide Networking went into somewhat more detail, particularly on the security policy aspects. It was presented at Health Care 96.
• Problems with the NHS Cryptography Strategy points out a number of errors in, and ethically unacceptable consequences of, a report on cryptography produced for the Department of Health. These comments formed the BMA's response to that report.

In 1996, the Government set up the Caldicott Committee to study the matter. Their report made clear that the NHS was already breaking confidentiality law by sharing data without consent; but the next Government just legislated (and regulated, and again) to give itself the power to share health data as the Secretary of State saw fit. (We objected and pointed out the problems the bill could cause; similar sentiments were expressed in a BMJ editorial, and a Nuffield Trust impact analysis, and BMJ letters here and here. Ministers claimed the records were needed for cancer registries: yet cancer researchers work with anonymised data in other countries – see papers here and here.) There was a storm of protest in the press: see the Observer, the New Statesman, and The Register. But that died down; the measure has now been consolidated as sections 251 and 252 of the NHS Act 2006, the Thomas-Walport review blessed nonconsensual access to health records (despite FIPR pointing out that this was illegal – a view later supported by the European Court). A government committee, the NHS Information Governmance Board, was set up oversee this lawbreaking, and Dame Fiona is being wheeled out once more. Centralised, nonconsensual health records not only contravene the I v Finland judgement but the Declaration of Helsinki on ethical principles for medical research and the Council of Europe recommendation no R(97)5 on the protection of medical data.

Two health IT papers by colleagues deserve special mention. Privacy in clinical information systems in secondary care describes a hospital system implementing something close to the BMA security policy (it is described in more detail in a special issue of the Health Informatics Journal, v 4 nos 3-4, Dec 1998, which I edited). Second, Protecting Doctors' Identity in Drug Prescription Analysis describes a system designed to de-identify prescription data for commercial use; although de-identification usually does not protect patient privacy very well, there are exceptions, such as here. This system led to a court case, in which the government tried to stop its owner promoting it – as it would have competed with their (less privacy-friendly) offerings. The government lost: the Court of Appeal decided that personal health information can be used for research without patient consent, so long as the de-identification is done competently.

Resources on what's happening in the USA include many NGOs: Patient Privacy Rights may have been the most influential, but see also EPIC, the Privacy Rights Clearinghouse, the Citizens' Council on Health Care, the Institute for Health Freedom. and CDT. Older resources include an NAS report entitled For the Record: Protecting Electronic Health Information, a report by the Office of Technology Assessment, a survey of the uses of de-identified records for the DHHS, and a GAO report on their use in Medicare. As for the basic science, see my book chapters on Boundaries and on Inference Control.

I chair the Foundation for Information Policy Research, the UK's leading Internet policy think tank, which I helped set up in 1998. We are not a lobby group; our enemy is ignorance rather than the government of the day, and our mission is to understand IT policy issues and explain them to policy makers and the press. We had a conference for our 25th anniversary in 2023 (blog), another for our 20th in 2018 (blog), and here are the issues as we saw them in 2008 and 1999. Some highlights of our work follow.

• Thirty Years of Crypto Wars: the great result of 2023 was that we beat the Chat Control proposal in the European Parliament. This involved dozens of NGOs lobbying for over a year backed by academics from a number of countries. One of my contributions was Chat Control or Child Protection, which analyses the arguments used by GCHQ that they should circumvent the end-to-end crypto in messenger apps "to protect children" and shows that they are not consistent with the evidence; and Bugs in our Pockets: The Risks of Client-Side Scanning, a technical study of the risks involved in mandatory scanning of people's phones and other devices for illegal materials, as proposed in various forms by the US and UK governments, the EU and originally Apple, who have at least had the sense to recant (blog). But the fight continues. One Protocol to Rule Them All? On Securing Interoperable Messaging analyses the EU DMA mandate for messaging systems interoperability. This will vastly increase the attack surface at every level in the stack (blog).
• That in turn updated a 2015 paper on the same topic, Keys Under Doormats, which argues that the push by the UK and US governments for exceptional access to all computer and communications data is wrong in principle and unworkable in practice (see also this video and this followup).
• In 2016, we organised the tenth Scrambling for Safety workshop on their Investgatory Powers Bill while it was on its way through Parliament. The chaos after the Brexit vote, plus May's appointment as Prime Minister, allowed this bill to get through Parliament unscathed. The European Court of Justice has already found that its data retention provisions contravene human rights but the government ignored this, and the Australian government followed suit.
• What Goes Around Comes Around is a chapter I wrote for a book by EPIC, on whose advisory board I sit.
• I first got engaged in technology policy thanks to attempts in the 1990s by governments to control the use of cryptography. In 1995, I wrote Crypto in Europe – Markets, Law and Policy, the first paper to point out that law enforcement communications intelligence was mostly about traffic analysis and criminal communications security was mostly traffic security. The Risks of Key Recovery, Key Escrow, and Trusted Third-Party Encryption became the most widely-cited publication on key escrow; it was originally presented as testimony to the US Senate, and then also to the Trade and Industry Committee of the UK House of Commons, together with a further piece I wrote, The Risks and Costs of UK Escrow Policy.
• The GCHQ Protocol and its Problems pointed out a number of serious defects in the protocol that the British government used to secure its electronic mail. Our analysis stopped the protocol being more widely adopted; the government is still trying to push its successor, which still suffers much the same problems. The government also proposed mandatory licensing of certification authorities, so we compiled The Global Trust Register – a certification authority implemented in paper and ink rather than electronics. Our book would have been banned by the new law – which enabled us to visit Culture Secretary Chris Smith at a critical point and get it on the Cabinet agenda. What we achieved with this campaign was to limit the scope of the Regulation of Investigatory Powers Act. Originally this would have allowed the police to obtain, without warrant, a complete history of everyone's web browsing activity, as ‘communications data’. Our ‘Big Browser amendment’got the House of Lords to limit this to the identity of the machines involved in a communication, rather than the full URLs. But the RIP Act still made it into law and has had a number of the bad effects we predicted.
• These issues revived in the 2000s with GCHQ's Interception Modernisation Programme, a plan to centralise all traffic data first in a central database (under Blair and Brown) and then in a system of federated databases maintained by communications service providers. FIPR wrote various papers on related matters, and when the Coalition Government brought its Communications Data Bill, we organised resistance. The bill was dropped after the Lib Dems finally vetoed it.
• Privacy has come under attack not just from the spooks but from the world of Big Data.The collection, linking and use of data in biomedical research and health care: ethical issues is a report we wrote for the Nuffield Bioethics Foundation: what happens to health privacy in a world with cloud-based medical records and pervasive genomics? (blog Guardian Indy Science).

  In 2009, our Database State report on the failings of public-sector IT in Britain, and how to fix them, got massive press coverage: the BBC, the Guardian (also here), the Mail (also here), the Independent, the Telegraph and Liberty Central. This followed an earlier ICO report on children's databases. Both the Lib Dems and the Conservatives promised to kill or change at least some of these systems; after they won power in the 2010 election their coalition agreement spelled the end of the ContactPoint children's database, and of ID cards. The subsequent review by my FIPR colleague Eileen Munro also sealed the fate of eCAF, another central children's database system.

• Sustainability interacts in various ways with information security, notably in the sustainability of software; but see also my talk on Privacy for Tigers.
• Brexit affects us in numerous ways. Brexit and technology explained how the Brexit debate largely ignored network externalities, which could make the damage worse. Brexit and Cambridge assesses the likely costs to the University (blog posts).
• Waste of Public Money is another objection to the bad government systems that undermine our privacy. Other wasteful systems include smart meters which look set to cost billions without achieving anything useful (blog).
• Identity Cards were a clever political move by Blair; they divided the Conservatives, so Blair promised to do them for almost a decade and never got round ot it. I testified to the Home Affairs committee in 2004 that they would not work as advertised, and contributed to the LSE Report that spelled this out in detail. I wrote various previous pieces in response to government identity consultations, on aspects such as smartcards and PKI.
• Internet Censorship is a growing problem, and not just in developing countries; I've been on the receiving end more than once. In 1995, I invented the first censorship-resistant system, the Eternity Service; this was a precursor of later file-sharing systems (see above), and we've also written on the economics of censorship resistance. But despite the technical difficulties and collateral costs of content filtering, governments aren't giving up. From 2006 to 2008, I was a principal investgator for the OpenNet Initiative which monitors Internet filtering worldwide. Shifting Borders reviewed the state of play in late 2007, and appeared in Index on Censorship. Tools and Technology of Internet Filtering goes into more technical detail. The political action now is about Internet blocking.
• Consumer Protection: FIPR also brought together legal and computing experts to deconstruct the fashionable late-1990s notion that ‘digital certificates’ would solve all the problems of e-commerce and e-government. Anyone inclined to believe such nonsence should read Electronic Commerce – Who Carries the Risk of Fraud?. Other work in this thread include FIPR's responses to consultations on smartcards, the electronic signature directive and the ecommerce bill.

  More recently we have seen the erosion of consumer rights as a result of the introduction of chip and PIN cards. The technical sections above describe how frauds happen; the flip side of the story is how the banks escape liability. Our analysis of the failings of the Financial Ombudsman Service remains unanswered; see also FIPR's submission on Personal Internet Security (with which the House of Lords basically agreed) and the National Payments Plan. FIPR now takes the view that the only way to fix consumer protection is to replace public action with private action, by changing the rules on costs so that consumers can enforce their rights in court without risking horrendous costs orders if they lose.

• Export Control: In 2001-02, FIPR persuaded the Lords to amend the Export Control Bill. This bill was designed to give ministers the power to license intangible exports. It was the result of US lobbying of Tony Blair in 1997; back then, UK crypto researchers could put source code on our web pages while our US colleagues weren't allowed to. In its original form, its provisions were so broad that it would have given ministers the power of pre-publication review of scientific papers. We defeated the Government in the House of Lords by 150-108, following a hard campaign – see press coverage in the BBC, the New Scientist, the Guardian and the Economist, and an article on free speech I wrote for IEEE Computing. But the best quote I have is also the earliest. The first book written on cryptology in English, by Bishop John Wilkins in 1641, remarked that ‘If all those useful Inventions that are liable to abuse, should therefore be concealed, there is not any Art or Science which might be lawfully profest’

  This issue revived in 2003, with a government attempt to wrest back by regulation much of what they conceded in parliament. FIPR fought back and extracted assurances from Lord Sainsbury about the interpretation of regulations made under the Act. Without our campaign, much scientific collaboration would have become technically illegal, leaving scientists open to arbitrary harrassment. Much credit goes to the Conservative frontbencher Doreen Miller, Liberal Democrat frontbencher Margaret Sharp, and the then President of the Royal Society Bob May, who made his maiden speech in the Lords on the issue and marshalled the crossbenchers. We are very grateful for their efforts.

• Trusted Computing was a focus in 2002-03. I wrote a Trusted Computing FAQ, followed by a study of the competition policy aspects which led inter alia to a symposium organised by the German government that pushed the Trusted Computing Group into incorporating. Microsoft couldn't get remote attestation to work; Intel abandoned trusted computing; and its only direct descendants were bitlocker and Arm's TrustZone.
• IP Enforcement: Our lobbying priority in 2003-04 was the EU IPR enforcement directive, which has been criticised by distinguished lawyers. Our lobbying got it amended to remove criminal sanctions for patent infringement and legal protection for devices such as RFID tags. This law was supported by the music industry, the luxury brands, and (initially) Microsoft, while the coalition that we put together to oppose it included the phone companies, the supermarkets, the generic drugmakers, the car parts industry, smaller software firms and the free software community. The press was sceptical – in Britain, France and even America. The issue was even linked to a boycott of Gillette. There is more on my blog.

  This was a watershed in copyright history: the IP lobby was never going to be stopped by fine words, only by another lobby pushing in the other direction, and the Enforcement Directive was when that first came together. It also led to the birth of EDRI, European Digital Rights, a confederation of European digital-rights NGOs, whose establishment was one of FIPR's significant achievements. EDRI's first campaign was against the IP Enforcement Directive; afterwards FIPR and EDRI established a common position on intellectual property. Since then I have given evidence to the Gowers Review of IP and a parliamentary committee on DRM. The lead UK NGO on IP nowadays is the Open Rights Group.

• Terrorism: Here are Comments on Terrorism I wrote after the 11th September attacks. The resulting hysteria made me work harder at developing security economics to enable policymakers and others to think more rationally about such things, once gthey calmed down. In the dark years that followed, I testified against police attempts to increase pre-charge detention to ninety days; and here is a video I did on the effects of 9/11. We must constantly push back on the scaremongers.

I served on Council, Cambridge University's governing body, 2003–10 and from 2015–18. I stood for election because of a proposal that most of the intellectual property generated by faculty members – from patents on bright ideas to books written up from lecture notes – would belong to the university rather than to its creator. To stop this, and to prevent further incidents like this one), we founded the Campaign for Cambridge Freedoms. The final vote approved a policy according to which academics keep copyright but the University gets a share of patent royalties. I got re-elected in 2006, and in my second term we won an important vote to protect academic freedom. For more, see my article from the Oxford Magazine. From 2013-4 I was on our Board of Scrutiny. In my third term my main contribution was investigating the delays and cost overruns in a large construction project.

Since then the culture wars came to Cambridge. Should our university require us to treat foolish or obnoxious colleagues with "respect", or just with "tolerance"? Our VC demanded "respect" but we called a free speech vote and academics voted decisively for tolerance instead. See Varsity, Newsweek, the FT, the Spectator, the Mail, the Sunday Times, the Times Higher Education Supplement, the Cambridge Student, the Cambridge Radical Feminist Network, Stephen Fry – and the Minister of State for Universities.

Our latest campaign is against Cambridge's policy of forcing academics to retire at 67, an outdated policy to which only Cambridge and Oxford cling; Oxford's version was found illegal in March 2023. Our campaign page is here.

My CV is here while my h-index is tracked here. I'm a Fellow of Churchill College, the Royal Society, the Royal Academy of Engineering, the Institution of Engineering and Technology, the Institute of Mathematics and its Applications, and the Institute of Physics. I won the 2015 Lovelace medal; the interviews I did for that award are here, while my oral history interview transcript is here and an Academy video is here. As for my academic genealogy, my thesis adviser was Roger Needham; his was Maurice Wilkes; then it runs back through Jack Ratcliffe, Edward Appleton, Ernest Rutherford, JJ Thomson, Lord Rayleigh, Edward Routh, William Hopkins, Adam Sedgwick, Thomas Jones, Thomas Postlethwaite, Stephen Whisson, Walter Taylor, Roger Smith, Roger Cotes, Isaac Newton, Isaac Barrow and Vincenzo Viviani to Galileo Galilei. For context, see my Unauthorised History of Cambridge University

Finally, here is my PGP key. If I revoke this key, I will always be willing to explain why I have done so provided that the giving of such an explanation is lawful. (For more, see FIPR.)

Security engineering is about building systems to remain dependable in the face of malice, error or mischance. As a discipline, it focuses on the tools, processes and methods needed to design, implement and test complete systems, and to adapt existing systems as their environment evolves. My book has become the standard textbook and reference since it was published in 2001. You can download both the first and second editions without charge here; the third edition will become free from 2024.

Security engineering is not just concerned with infrastructure matters such as firewalls and PKI. It's also about specific applications, such as banking and medical record-keeping, and increasingly about embedded systems such as payment terminals and burglar alarms. It's usually done badly, so it often takes several attempts to get a design right. It's also hard to learn: although there were good books on a number of the component technologies, such as cryptography and operating systems, there was little about how to use them effectively, and even less about how to make them work together. Most systems don't fail because the mechanisms are weak, but because they're used wrong.

My book was an attempt to help the working engineer to do better. As well as the basic science, it contains details of many applications – and lot of case histories of how their protection failed. It describes a number of technologies which aren't well described elsewhere. The first edition was pivotal in founding the now-flourishing field of information security economics: I realised that the narrative had to do with incentives and organisation at least as often as with the technology. The second edition incoporated the economic perspectives we've developed since then, and new perspectives from the psychology of security, as well as updating the technological side of things. The third edition is an update for the new world of phones, cloud services and social media; it tackles the problems raised by cars and medical devices such as the interaction of security with safety, and the costs of long-term patching; it also adds a huge amount about modern threat actors, from the cybercrime ecosystem to what we learned about state capabilities from the Snowden leaks and elsewhere.

More ...

2022 highlights include ExtremeBB, a database we collect of extremist postings to support research by political scientists, criminologists and others; CoverDrop which lets a newspaper build an end-to-end encrypted messenger into its app for whistleblowers; a paper on Chat Control or Child Protection for the latest round of the crypto wars; a study of the failures of security proofs; and two developments of Bad Characters and Trojan Source – one showing how these techniques easily mislead search engines, while the other mapping the impulse response of the vulnerability disclosure ecosystem.

2021 highlights include Bad characters and Trojan source, of which the first broke all large language models and the second all computer languages; two adversarial machine-learning papers, on data ordering attacks and markpainting; an analysis of cybercrime ventures as startups; and Bugs in our Pockets, the latest round in the Crypto Wars.

2020 highlights include sponge attacks and nudge attacks on machine-learning systems, along with work on adversarial reinforcement learning and on decoding smartphone sounds with a voice assistant. But my main project in 2020 was writing a third edition of my Security Engineering textbook.

2019 highlights include an acoustic side channel on smartphones, one paper on whistleblowing and two papers on blocking adversarial machine learning. The big paper was on Measuring the Changing Cost of Cybercrime; since we did the first systematic study seven years ago, the patterns changed surprisingly little despite a huge changed in technology. Finally I gave an invited talk at 36C3 on the sustainability of safety, security and privacy.

2018 highlights include papers on what's wrong with bitcoin exchanges and how to trace stolen bitcoin; on making security sustainable; controlling side effects in mainstream C compilers; how protocols evolve and a gullibility metric. There's also an invited talk on privacy for tigers.

2017 highlights include Standardisation and Certification in the Internet of Things, an analysis for the EU of what happens when we get software everywhere, and which informed EU directive 2019/771 on the sale of goods; DigiTally, a prototype payment system we built to extend mobile phone payments to areas of less developed countries with no phone service, using a novel protocol; and a book chapter I wrote for EPIC.

2016 highlights include a new Android side channel; an investigation of the social externalities of trust; studies of when lying feels the right thing to do, of taking down websites to prevent crime and bank fraud reimbursement; and finally two papers on Brexit.

2015 highlights included Keys Under Doormats, on what's wrong with government attempts to mandate exceptional access to all our data; a Nuffield report on what happens to health privacy in a world of cloud-based medical records and pervasive genomics; a report on the emotional impact of Internet fraud; two papers on how to do lie detection using analysis of body motion; severe flaws in Android factory reset and mobile anti-virus apps; and a novel demonstration that the Bell test results can come from pre-existing long-range order as easily as from nonlocal action.

2014 highlights included papers on Chip and Skim describing pre-play frauds against EMV bank cards; Security protocols and evidence which explains how the systems needed to support proper dispute resolution just don't get built; Experimental Measurement of Attitudes Regarding Cybercrime, on how prosecutors and public opinion are out of step; The psychology of malware warnings, on how to get users to pay attention to risk; Privacy versus government surveillance, on network economics and international relations; and Why bouncing droplets are a pretty good model of quantum mechanics, which solves an outstanding mystery in physics.

2013 highlights included Rendezvous, a prototype search engine for code; a demonstration that we could steal your PIN via your phone camera and microphone; an analysis of SDN Authentication; and papers on quantum computing and Bell's inequality.

2012 highlights included a big report on Measuring the Cost of Cybercrime and a history of security economics; an attempt to kill the government's smart metering project; three papers on dynamic networks; and four papers on payment protocols: Chip and Skim: cloning EMV cards with the pre-play attack, How Certification Systems Fail, A birthday present every eleven wallets? and Social Authentication – harder than it looks. Finally, Risk and privacy implications of consumer payment innovation discusses both payment and economic issues.

2011 highlights included a major report on the Resilience of the Internet Interconnection Ecosystem which studies how an attacker might bring down the Internet; an updated survey paper on Economics and Internet Security which covers recent analytical, empirical and behavioral research; and Can We Fix the Security Economics of Federated Authentication? which explores how we can deal with a world in which your mobile phone contains your credit cards, your driving license and even your car key. What happens when it gets stolen or infected? (blog)

2010 highlights included a paper on why Chip and PIN is broken for which we got coverage on Newsnight and a best paper award (later, the banks tried to suppress this research). Other bank security work included a paper on Verified by VISA and another on the unwisdom of banks adopting proprietary standards. On the control systems front, we published papers on the technical security and security economics of smart meters, on their privacy, on their deployment and on key management for substations. I created a psychology and security web page and wrote a paper on putting context and emotion back in security decisions.

2009 highlights included Database State, a report we wrote about the failings of public-sector IT – many of whose recommendations were adopted by the government elected in 2010; The snooping dragon which explains how the Chinese spooks hacked the Dalai Lama in the run-up to the Peking Olympics; Eight Friends are Enough, which shows how little privacy you have on Facebook; and The Economics of Online Crime. There's a videos of a talk I gave on dependability at the IET, as well as a survey paper, the slides, and a podcast. Finally, I wrote an Unauthorised History of Cambridge University.

2008 highlights included a major study of Security Economics and European Policy for the European Commission; the second edition of my book "Security Engineering"; the discovery of serious vulnerabilities in Chip and PIN payment systems; an analysis of the failings of the Financial Ombudsman Service (see also a video from the World Economic Forum in November 2008); the FIPR submission to the Thomas-Walport Review; a piece on confidentiality in the British Journal of General Practice; three videos on privacy made by ARCH; and a video on surveillance. I started a Workshop on Security and Human Behaviour to bring together psychologists with economists and security engineers to work on deception and risk.

2007 highlights included technical papers on RFID and on New Strategies for Revocation in Ad-Hoc Networks (which explores when suicide attacks are effective); a paper on fraud, risk and nonbank payment systems I wrote for the Fed; and a survey paper on Information Security Economics (of which a shortened version appeared in Science). I was a special adviser to House of Commons Health Committee for their Report on the Electronic Patient Record. Finally, following the HMRC data loss, I appeared in the debate on Newsnight.

2006 highlights included technical papers on topics from protecting power-line communications to the Man-in-the-Middle Defence, as well as a major report on the safety and privacy of children's databases for the Information Commissioner. I ended the year debating health privacy with health minister Lord Warner.

2005 highlights included research papers on The topology of covert conflict, on combining cryptography with biometrics, on Sybil-resistant DHT routing, and on Robbing the bank with a theorem prover; and a big survey paper on cryptographic processors.

2004 highlights included papers on cipher composition, key establishment in ad-hoc networks and the economics of censorship resistance. I also lobbied for amendments to the EU IP Enforcement Directive and organised a workshop on copyright which led to a common position adopted by many European NGOs.

I only write and referee for open publications, so I discard emails asking for reports for journals that sit behind a paywall.

By default, when I post a paper here I license it under the relevant Creative Commons license; you may redistribute it with attribution but not modify it.

I can no longer admit PhD students for Cambridge, because of forthcoming mandatory retirement; so if you want to do a PhD, please read the relevant web pages. I still admit PhD students at Edinburgh.