Software and Security Engineering

As software and communications become embedded invisibly everywhere, safety and security are becoming increasingly intertwined. The disciplines of software engineering and security engineering are converging. This course attempts a unified introduction.

Lecture materials

Copies of the lecture slides as printed.

Slides, with annotated notes, will appear here after each lecture:

Lecture 1 (26th April)
Lecture 2 (29th April)
Lecture 3 (1st May)

Supervision work

Examples sheet
Examples sheet solutions (supervisors only)

Past exam questions

Past exam questions since 2017 are here. Before that, questions from the software engineering component of the course are here and here, while for supervisions in the security component of the course, you might try previous exam questions on car locks, phone scratchcards, exam security, prospect theory, secret key protocols, public key protocols and banking authentication. Finally, if you want to get your teeth into the subject, you might research and write a case study of another software failure that caused substantial damage.

Lectures by Ross Anderson and further materials

Unfortunately Ross Anderson died unexpectedly at the end of March. We therefore have a diffent roster of lecturers this term which cover the same material and syllabus. Recordings of Ross's lectures alongside the further materials are available here.

You should probably budget 3-4 hours for each lecture, to watch the presentations and work through the supplementary material.

Lecture 1

The Introduction sets out the course's aims and provides an outline (18 minutes);
Definitions of terms such as security, safety, system, and trust (23 minutes);
The third segment explores Three information-flow security policies (19 minutes);

Basic reading is the first chapter of Security Engineering; see also the video Hackers remotely kill a jeep on the highway, and the story behind it. For additional background reading on multilevel security and safety policies is covered in chapter nine of Security Engineering.

Lecture 2

The second lecture starts with a discussion of how we develop a Safety policy (8 mins);
Next we explore policies for Separation of duty in the context of bookkeeping (14 mins);
Then, please watch this video of a crash test (2 mins), which leads in to Safety and security usability, and the pyramid of harms (13 minutes);
Now look at this video on the poor safety usability of syringe pumps (2 mins), which leads to the Safety usability of medical devices (7 minutes).

Basic reading about online harms is the book chapter Who is the Opponent?, while for separation of duty it's that on Banking and bookkeeping pp 376-393 (the rest of the chapter deals with payment systems, and is relevant to lecture 4). Harold Thimbleby's paper on safety usability failures in medical devices is here.

Lecture 3

We first study the Types of error people make and the error rates (14 minutes);
Next is Social psychology which gives us many useful insights into authority, conformity, marketing and fraud (15 minutes);
This leads to a discussion of Mental models, affordances, defaults and discrimination (12 minutes);
Our final security psychology topic is Passwords. Think of a phrase you can't remember and don't write it down! (18 mins)

Basic reading is th book chapter on Psychology and Usability. Several series of TV programmes were made in The Real Hustle series, showing how scams work in practice; there's a summary by Paul Wilson, one of the stars of that series, and our own Professor Frank Stajano here. Many of the same principles apply to the communication of health risks. Why Johnny Can't Encrypt is a classic paper that kicked off research on security usability. You may also want to review the seminal experiments by Solomon Asch, Stanley Milgram and Philip Zimbardo. Finally, Mohammed Aamir Ali's paper is here, and here's Mat Honan's story.

Lecture 4

Our first security protocol is Ordering wine in a restaurant (2 minutes);
Next we look at Remote key entry protocols for cars (9 minutes);
Then it's Identify-friend-or-foe and the man-in-the-middle attack (4 minutes);
Then we discuss Attacks on two-factor authentication (4 minutes);
Two of the most widely-used protocols are Kerberos and EMV (8 minutes);
Here are some Simple attacks on EMV (5 minutes);
Next please watch this piece we did for the BBC on the No-PIN attack (9 minutes);
Finally we explain the No-PIN and preplay attacks (7 minutes).

Basic reading is the book chapter on Protocols, and for further reading on payment fraud there's the rest of the chapter on Banking and bookkeeping. For fun here's a chip and PIN terminal playing Tetris.

Lecture 5:

We start public-key crypto revision with the tale of Anthony, Brutus, Caesar and the dispatch box (3 minutes);
Next we revise public-key encryption (6 minutes);
We look at middleperson attacks on protocols, then describe SSL/TLS (9 minutes);
This leads to a discussion of What goes wrong with SSL/TLS (8 minutes);
We then start discussing different types of software bug, with our first case study being the Patriot missile (11 minutes);
One of the most famous security bugs was Heartbleed (6 minutes);
There follows an introduction to Code injection (4 minutes);
And we finish by discussing the classic attack, Buffer overflows (8 minutes).

Basic reading on public-key crypto is in Chapter 5 pages 185–203. Here's the Need for a Boeing 787 reboot, The Bug Heard Round the World, Heartbleed, the Whopper Burger ad and its back story. Some further hacks of possible interest are here, here and here. For the keen, there's more on malware in my book Chapter 21.

Lecture 6

Here's the Introduction (3 minutes);
a description of the London Ambulance Service disaster, a flashbulb moment in the history of software engineering (25 minutes);
Then we talk about the NHS National Programme for IT, which was for some years the most expensive civilian IT project disaster (6 minutes);
Then it's the turn of Smart meters, which may cost even more (8 minutes);
And finally Universal credit, a system whose development was highly problematic but which thankfully survived enormous strain during the pandemic (5 minutes).

The basic reading is the report of the inquiry into the London Ambulance System disaster; for further reading, the case study of the NHS National Programme for IT is here, there are links to papers on the smart meter project here, and the National Audit Office report into Rolling out Universal Credit is perhaps the best starting point for that story.

Lecture 7

First, Software engineering is about managing complexity (12 minutes);
We next discuss Software engineering economics (18 minutes);
We then go through the Waterfall model (13 minutes);
And then there's a quick introduction to Iterative models of development (8 minutes).

Here's Fred Brooks' article No Silver Bullet; there's also a piece I recorded with Stephen Fry on Y2K.

Lecture 8

We start off with an Introduction to critical systems (2 minutes);
Please then watch the video of the Tacoma Narrows bridge collapse (5 minutes);
Safety is an emergent system property (7 minutes);
The Therac-25 accidents illustrate how not to do safety engineering (10 minutes);
We then discuss the difficulties of managing redundancy (11 minutes);
This sets us up to discuss the world's worst ever software failure, that of the Boeing 737 Max (13 minutes);
And finally we discuss the overall process of safety engineering (8 minutes).

The first key primary source is Nancy Leveson's paper on The Therac-25 accidents, while an article from the New York Times documents how fatalities continue to be caused by poor radiology software. Please also watch this video on the Boeing 737 Max crashes. Here is the report of a Qantas flight where the plane's three onboard computers started arguing with each other, and here is the report on oscillations in London's Millennium Bridge.

Lecture 9

We start off with a discussion of Tools and methods (12 minutes);
The second topic is Individual versus group productivity (5 minutes);
Next is the Evolution of testing and agile development (15 minutes);
We then discuss Post-market surveillance and coordinated disclosure (9 minutes);
Next is whether we focus on outcomes or process (5 minutes);
This leads to a discussion of Project management (18 minutes);
And finally to our Conclusions (10 minutes).

The Coverity paper is A Few Billion Lines of Code Later; here's Eric Raymond's essay The Cathedral and the Bazaar; and here's the paper by Curtis, Krasner and Iscoe on how large projects fail. Finally, here's a piece on software sustainability, and a talk on the Sustainability of Safety, Security and Privacy – if you didn't watch it at the start of the course!

Further and background reading includes Building Secure and Reliable Systems by six Googlers – Heather Adkins, Betsy Beyer, Paul Blankinship, Ana Oprea, Piotr Lewandowski and Adam Stubblefield. While Ross's textbook reflects his experience with payment systems, healthcare systems and other distributed systems involving specialist devices, this book's focus is on developing and maintaining large websites and cloud systems generally. It gives a great overview of the interaction between dependability and security in that environment. (You should be able to get free access through the university library.) The Netflix' A/B testing platform uses a more sophisticated technique called stratified sampling, and allows fine-grained control for how users are allocated to concurrently running tests; an A/B test Facebook ran that many would argue crossed the line ethically; the Kubernetes platform; and AWS, whose services to developers include not just Kubernetes and load balancing, but EC2 (low-level infrastructure management) and CloudFormation (an IaaC framework).

Here is a talk on the Sustainability of Safety, Security and Privacy which brings together a number of the themes of this course. Ross delivered it at 36C3, Europe's biggest security event, in Leipzig in December 2019. It's up to you whether you watch this at the start of the course, or the end!