Software and Security Engineering

As software and communications become embedded invisibly everywhere, safety and security are becoming increasingly intertwined. The disciplines of software engineering and security engineering are converging. This course attempts a unified introduction.

Here are the slides for the main lectures of the course. The slides for Dr Richard Sharp's guest lecture (lecture 8) are here. You should probably budget 3-4 hours for each lecture, to watch the presentations and work through the supplementary material.

I'm sorry that the lectures show last year's date; it didn't occur to me, when I recorded them in March 2020, that we'd still be in lockdown in April 2021 and teaching remotely. Re-recording them to change the date from 2020 to 2021 would be unreasonable, and it's departmental policy that last year's recordings can be re-used. I have made appropriate updates to the links in this page instead.

I will also hold three live classes, at 1100 BST on May 10, 19 and 26 (the time slots for lectures 5 and 9, and the Wednesday after lectures), to answer questions. Links will be circulated on the class mailing list.

Here is a talk on the Sustainability of Safety, Security and Privacy which brings together a number of the themes of this course. I gave it at 36C3, Europe's biggest security event, in Leipzig in December 2019. It's up to you whether you watch this at the start of the course, or the end!

Lecture 1 (please work though this by April 30):

• The Introduction sets out the course's aims and provides an outline (18 minutes);
• I then explore the Definitions of terms such as security, safety, system, and trust (23 minutes);
• The third segment explores Three information-flow security policies (19 minutes);

Basic reading is the first chapter of Security Engineering; see also the video Hackers remotely kill a jeep on the highway, and the story behind it. For additional background reading on multilevel security and safety policies, I might suggest chapter nine of Security Engineering.

Lecture 2 (May 3):

• The second lecture starts with a discussion of how we develop a Safety policy (8 mins);
• Next we explore policies for Separation of duty in the context of bookkeeping (14 mins);
• Then, please watch this video of a crash test (2 mins), which leads in to Safety and security usability, and the pyramid of harms (13 minutes);
• Now look at this video on the poor safety usability of syringe pumps (2 mins), which leads to the Safety usability of medical devices (7 minutes).

Basic reading about online harms is my book chapter Who is the Opponent?, while for separation of duty it's that on Banking and bookkeeping pp 376-393 (the rest of the chapter deals with payment systems, and is relevant to lecture 4). Harold Thimbleby's paper on safety usability failures in medical devices is here.

Lecture 3 (May 5):

• We first study the Types of error people make and the error rates (14 minutes);
• Next is Social psychology which gives us many useful insights into authority, conformity, marketing and fraud (15 minutes);
• This leads to a discussion of Mental models, affordances, defaults and discrimination (12 minutes);
• Our final security psychology topic is Passwords. Think of a phrase you can't remember and don't write it down! (18 mins)

Basic reading is my book chapter on Psychology and Usability. Several series of TV programmes were made in The Real Hustle series, showing how scams work in practice; there's a summary by Paul Wilson, one of the stars of that series, and our own Professor Frank Stajano here. Many of the same principles apply to the communication of health risks. Why Johnny Can't Encrypt is a classic paper that kicked off research on security usability. You may also want to review the seminal experiments by Solomon Asch, Stanley Milgram and Philip Zimbardo. Finally, Mohammed Aamir Ali's paper is here, and here's Mat Honan's story.

Lecture 4 (May 7):

• Our first security protocol is Ordering wine in a restaurant (2 minutes);
• Next we look at Remote key entry protocols for cars (9 minutes);
• Then it's Identify-friend-or-foe and the man-in-the-middle attack (4 minutes);
• Then we discuss Attacks on two-factor authentication (4 minutes);
• Two of the most widely-used protocols are Kerberos and EMV (8 minutes);
• Here are some Simple attacks on EMV (5 minutes);
• Next please watch this piece we did for the BBC on the No-PIN attack (9 minutes);
• Finally we explain the No-PIN and preplay attacks (7 minutes).

Basic reading is my book chapter on Protocols, and for further reading on payment fraud there's the rest of the chapter on Banking and bookkeeping. For fun here's a chip and PIN terminal playing Tetris.

Lecture 5 (May 10):

• We start public-key crypto revision with the tale of Anthony, Brutus, Caesar and the dispatch box (3 minutes);
• Next we revise public-key encryption (6 minutes);
• We look at middleperson attacks on protocols, then describe SSL/TLS (9 minutes);
• This leads to a discussion of What goes wrong with SSL/TLS (8 minutes);
• We then start discussing different types of software bug, with our first case study being the Patriot missile (11 minutes);
• One of the most famous security bugs was Heartbleed (6 minutes);
• There follows an introduction to Code injection (4 minutes);
• And we finish by discussing the classic attack, Buffer overflows (8 minutes).

Basic reading on public-key crypto is my book Chapter 5 pages 185–203. Here's the Need for a Boeing 787 reboot, The Bug Heard Round the World, Heartbleed, the Whopper Burger ad and its back story. Some further hacks of possible interest are here, here and here. For the keen, there's more on malware in my book Chapter 21.

Lecture 6 (May 12):

• Here's the Introduction (3 minutes);
• Here I describe the London Ambulance Service disaster, a flashbulb moment in the history of software engineering (25 minutes);
• Then we talk about the NHS National Programme for IT, which was for some years the most expensive civilian IT project disaster (6 minutes);
• Then it's the turn of Smart meters, which may cost even more (8 minutes);
• And finally I talk about Universal credit, a system whose development was highly problematic but which thankfully survived enormous strain during the pandemic (5 minutes).

The basic reading is the report of the inquiry into the London Ambulance System disaster; for further reading, the case study of the NHS National Programme for IT is here, there are links to papers on the smart meter project here, and the National Audit Office report into Rolling out Universal Credit is perhaps the best starting point for that story.

Lecture 7 (May 14):

• First, Software engineering is about managing complexity (12 minutes);
• We next discuss Software engineering economics (18 minutes);
• We then go through the Waterfall model (13 minutes);
• And then there's a quick introduction to Iterative models of development (8 minutes).

Here's Fred Brooks' article No Silver Bullet; there's also a piece I recorded with Stephen Fry on Y2K.

Lecture 8 – guest lecture by Dr Richard Sharp (May 17):

• The economics of software as a Service (SaaS) (13 minutes);
• The impact SaaS has on software engineering (28 minutes);
• Behavioural analytics and experiment frameworks (21 minutes).

For further reading, see Netflix' A/B testing platform, which uses a more sophisticated technique called stratified sampling, and allows fine-grained control for how users are allocated to concurrently running tests; an A/B test Facebook ran that many would argue crossed the line ethically; the Kubernetes platform; and AWS, whose services to developers include not just Kubernetes and load balancing, but EC2 (low-level infrastructure management) and CloudFormation (an IaaC framework).

Lecture 9 (May 19):

• We start off with an Introduction to critical systems (2 minutes);
• Please then watch the video of the Tacoma Narrows bridge collapse (5 minutes);
• Safety is an emergent system property (7 minutes);
• The Therac-25 accidents illustrate how not to do safety engineering (10 minutes);
• We then discuss the difficulties of managing redundancy (11 minutes);
• This sets us up to discuss the world's worst ever software failure, that of the Boeing 737 Max (13 minutes);
• And finally we discuss the overall process of safety engineering (8 minutes).

The first key primary source is Nancy Leveson's paper on The Therac-25 accidents, while an article from the New York Times documents how fatalities continue to be caused by poor radiology software. Please also watch this video on the Boeing 737 Max crashes. Here is the report of a Qantas flight where the plane's three onboard computers started arguing with each other, and here is the report on oscillations in London's Millennium Bridge.

Lecture 10 (May 21):

• We start off with a discussion of Tools and methods (12 minutes);
• The second topic is Individual versus group productivity (5 minutes);
• Next is the Evolution of testing and agile development (15 minutes);
• We then discuss Post-market surveillance and coordinated disclosure (9 minutes);
• Next is whether we focus on outcomes or process (5 minutes);
• This leads to a discussion of Project management (18 minutes);
• And finally to our Conclusions (10 minutes).

The Coverity paper is A Few Billion Lines of Code Later; here's Eric Raymond's essay The Cathedral and the Bazaar; and here's the paper by Curtis, Krasner and Iscoe on how large projects fail. Finally, here's a piece I wrote on software sustainability, and a talk I gave on the Sustainability of Safety, Security and Privacy – if you didn't watch it at the start of the course!

For further and background reading I recommend Building Secure and Reliable Systems by six Googlers – Heather Adkins, Betsy Beyer, Paul Blankinship, Ana Oprea, Piotr Lewandowski and Adam Stubblefield. While my book reflects my own experience with payment systems, healthcare systems and other distributed systems involving specialist devices, this book's focus is on developing and maintaining large websites and cloud systems generally. It gives a great overview of the interaction between dependability and security in that environment. (You should be able to get free access through the university library.)

Past exam questions for 2017-19 are here. Before that, questions from the software engineering component of the course are here and here, while for supervisions in the security component of the course, you might try previous exam questions on car locks, phone scratchcards, exam security, prospect theory, secret key protocols, public key protocols and banking authentication. Finally, if you want to get your teeth into the subject, you might research and write a case study of another software failure that caused substantial damage.

Last year’s course materials are still available.