Course pages 2019–20
Software and Security Engineering
As software and communications become embedded invisibly everywhere, safety and security are becoming increasingly intertwined. The disciplines of software engineering and security engineering are converging. This course attempts a unified introduction.
Here are the slides for the main lectures of the course. The slides for Dr Richard Sharp's guest lecture (lecture 8) are here. You should probably budget 3-4 hours for each lecture, to watch the presentations and work through the supplementary material.
Here is a talk on the Sustainability of Safety, Security and Privacy which brings together a number of the themes of this course. I gave it at 36C3, Europe's biggest security event, in Leipzig last December. It's up to you whether you watch this at the start of the course, or the end!
Lecture 1 (please work though this by April 24):
- The Introduction sets out the course's aims and provides an outline (18 minutes);
- I then explore the Definitions of terms such as security, safety, system, and trust (23 minutes);
- The third segment explores Three information-flow security policies (19 minutes);
Basic reading is the first chapter of Security Engineering; see also the video Hackers remotely kill a jeep on the highway, and the story behind it. For additional background reading on multilevel security and safety policies, I might suggest chapter nine of Security Engineering.
Lecture 2 (April 27):
- The second lecture starts with a discussion of how we develop a Safety policy (8 mins);
- Next we explore policies for Separation of duty in the context of bookkeeping (14 mins);
- Then, please watch this video of a crash test (2 mins), which leads in to Safety and security usability, and the pyramid of harms (13 minutes);
- Now look at this video on the poor safety usability of syringe pumps (2 mins), which leads to the Safety usability of medical devices (7 minutes).
Basic reading about online harms is my book chapter Who is the Opponent?, while for separation of duty it's that on Banking and bookkeeping pp 362-376 (the rest of the chapter deals with payment systems, and is relevant to lecture 4). Harold Thimbleby's paper on safety usability failures in medical devices is here.
Lecture 3 (April 29):
- We first study the Types of error people make and the error rates (14 minutes);
- Next is Social psychology which gives us many useful insights into authority, conformity, marketing and fraud (15 minutes);
- This leads to a discussion of Mental models, affordances, defaults and discrimination (12 minutes);
- Our final security psychology topic is Passwords. Think of a phrase you can't remember and don't write it down! (18 mins)
Basic reading is my book chapter on Psychology and Usability. Several series of TV programmes were made in The Real Hustle series, showing how scams work in practice; there's a summary by Paul Wilson, one of the stars of that series, and our own Professor Frank Stajano here. Many of the same principles apply to the communication of health risks. Why Johnny Can't Encrypt is a classic paper that kicked off research on security usability. You may also want to review the seminal experiments by Solomon Asch, Stanley Milgram and Philip Zimbardo. Finally, Mohammed Aamir Ali's paper is here, and here's Mat Honan's story.
Lecture 4 (May 1):
- Our first security protocol is Ordering wine in a restaurant (2 minutes);
- Next we look at Remote key entry protocols for cars (9 minutes);
- Then it's Identify-friend-or-foe and the man-in-the-middle attack (4 minutes);
- Then we discuss Attacks on two-factor authentication (4 minutes);
- Two of the most widely-used protocols are Kerberos and EMV (8 minutes);
- Here are some Simple attacks on EMV (5 minutes);
- Next please watch this piece we did for the BBC on the No-PIN attack (9 minutes);
- Finally we explain the No-PIN and preplay attacks (7 minutes).
Basic reading is my book chapter on Protocols, and for further reading on payment fraud there's the rest of the chapter on Banking and bookkeeping. For fun here's a chip and PIN terminal playing Tetris.
Lecture 5 (May 4):
- We start public-key crypto revision with the tale of Anthony, Brutus, Caesar and the dispatch box (3 minutes);
- Next we revise public-key encryption (6 minutes);
- We look at middleperson attacks on protocols, then describe SSL/TLS (9 minutes);
- This leads to a discussion of What goes wrong with SSL/TLS (8 minutes);
- We then start discussing different types of software bug, with our first case study being the Patriot missile (11 minutes);
- One of the most famous security bugs was Heartbleed (6 minutes);
- There follows an introduction to Code injection (4 minutes);
- And we finish by discussing the classic attack, Buffer overflows (8 minutes).
Basic reading on public-key crypto is my book Chapter 5 pages 176–190. Here's the Need for a Boeing 787 reboot, The Bug Heard Round the World, Heartbleed, the Whopper Burger ad and its back story. Some further hacks of possible interest are here, here and here. For the keen, there's more on malware in my book Chapter 21.
Lecture 6 (May 6):
- Here's the Introduction (3 minutes);
- Here I describe the London Ambulance Service disaster, a flashbulb moment in the history of software engineering (25 minutes);
- Then we talk about the NHS National Programme for IT, which was for some years the most expensive civilian IT project disaster (6 minutes);
- Then it's the turn of Smart meters, which may cost even more (8 minutes);
- And finally I talk about Universal credit, a system whose development has been highly problematic and which is now under enormous strain thanks to the pandemic (5 minutes).
The basic reading is the report of the inquiry into the London Ambulance System disaster; for further reading, the case study of the NHS National Programme for IT is here, there are links to papers on the smart meter project here, and the National Audit Office report into Rolling out Universal Credit is perhaps the best starting point for that story.
Lecture 7 (May 8):
- First, Software engineering is about managing complexity (12 minutes);
- We next discuss Software engineering economics (18 minutes);
- We then go through the Waterfall model (13 minutes);
- And then there's a quick introduction to Iterative models of development (8 minutes).
Here's Fred Brooks' article No Silver Bullet; there's also a piece I recorded with Stephen Fry on Y2K.
Lecture 8 – guest lecture by Dr Richard Sharp (May 11):
- The economics of software as a Service (SaaS) (13 minutes);
- The impact SaaS has on software engineering (28 minutes);
- Behavioural analytics and experiment frameworks (21 minutes).
For further reading, see Netflix' A/B testing platform, which uses a more sophisticated technique called stratified sampling, and allows fine-grained control for how users are allocated to concurrently running tests; an A/B test Facebook ran that many would argue crossed the line ethically; the Kubernetes platform; and AWS, whose services to developers include not just Kubernetes and load balancing, but EC2 (low-level infrastructure management) and CloudFormation (an IaaC framework).
Lecture 9 (May 13):
- We start off with an Introduction to critical systems (2 minutes);
- Please then watch the video of the Tacoma Narrows bridge collapse (2 minutes);
- Safety is an emergent system property (7 minutes);
- The Therac-25 accidents illustrate how not to do safety engineering (10 minutes);
- We then discuss the difficulties of managing redundancy (11 minutes);
- This sets us up to discuss the world's worst ever software failure, that of the Boeing 737 Max (13 minutes);
- And finally we discuss the overall process of safety engineering (8 minutes).
The first key primary source is Nancy Leveson's paper on The Therac-25 accidents, while an article from the New York Times documents how fatalities continue to be caused by poor radiology software. Please also watch this video on the Boeing 737 Max crashes. Here is the report of a Quantas flight where the plane's three onboard computers started arguing with each other, and here is the report on oscillations in London's Millennium Bridge.
Lecture 10 (May 15):
- We start off with a discussion of Tools and methods (12 minutes);
- The second topic is Individual versus group productivity (5 minutes);
- Next is the Evolution of testing and agile development (15 minutes);
- We then discuss Post-market surveillance and coordinated disclosure (9 minutes);
- Next is whether we focus on outcomes or process (5 minutes);
- This leads to a discussion of Project management (15 minutes);
- And finally to our Conclusions (10 minutes).
The Coverity paper is A Few Billion Lines of Code Later; here's Eric Raymond's essay The Cathedral and the Bazaar; and here's the paper by Curtis, Krasner and Iscoe on how large projects fail. Finally, here's a piece I wrote on software sustainability, and a talk I gave on the Sustainability of Safety, Security and Privacy – if you didn't watch it at the start of the course!
For further and background reading I recommend Building Secure and Reliable Systems by six Google engineers – Heather Adkins, Betsy Beyer, Paul Blankinship, Ana Oprea, Piotr Lewandowski and Adam Stubblefield. This book came out while I was preparing the course and so doesn't appear on the syllabus. It gives a great overview of the interaction between dependability and security in cloud systems, while my own book is more driven by my experience with embedded systems and specific applications.
Past exam questions for 2017-19 are here. Before that, questions from the software engineering component of the course are here and here, while for supervisions in the security component of the course, you might try previous exam questions on car locks, phone scratchcards, exam security, prospect theory, secret key protocols, public key protocols and banking authentication. Finally, if you want to get your teeth into the subject, you might research and write a case study of another software failure that caused substantial damage.