Psychology and Security Resource Page
Ross
Anderson
A fascinating dialogue is developing between psychologists and security
engineers. At the macro scale, societal overreactions to terrorism are founded
on the misperception of risk and uncertainty, which has deep psychological
roots. At the micro scale, more and more crimes involve deception; as security
engineering gets better, it's easier to mislead people than to hack computers
or hack through walls. Many systems also fail because of usability problems:
the designers have different mental models of threats and protection mechanisms
from users. Wrong assumptions about users can lead systems to discriminate
against women, the less educated and the elderly. And misperceptions cause
security markets to fail: many users buy snake oil, while others distrust
quite serviceable mechanisms. Security is both a feeling and a reality, and
they're different. The gap gets ever wider, and ever more important.
At a deeper level, the psychology of security touches on fundamental scientific
and philosophical problems. The `Machiavellian Brain' hypothesis states that we
evolved high intelligence not to make better tools, but to use other monkeys
better as tools: primates who were better at deception, or at detecting
deception in others, left more descendants. Conflict is also deeply tied up
with social psychology and anthropology, while evolutionary explanations for
the human religious impulse involve both trust and conflict. The dialogue
between researchers in security and in psychology has thus been widening,
bringing in people from usability engineering, protocol design, privacy, and
policy on the one hand, and from social psychology, evolutionary biology, and
behavioral economics on the other. We believe that this new discipline will
increasingly become one of the active contact points between computing and
psychology – an exchange that has hugely benefited both disciplines for
over a generation.
This page provides links to a number of key papers, workshops, the home pages of
active researchers, relevant books, and other resources. Complementary pages include my security economics resource
page and Alessandro
Acquisti's privacy economics page.
The most relevant regular event is the Security and Human Behaviour
workshop.
Introductory Papers
- In The Psychology of
Security, Bruce Schneier gives a layman's introduction to how heuristics
and biases affect the way we deal with risk and uncertainty. Our analysis of
security problems can thus draw on the large literature on behavioral economics
with its insights on risk aversion, the availability heuristic, mental
accounting, and discounting. (See also Bruce's other essays on
psychology and security.)
- For a more detailed treatment, see the chapter
on security and
psychology from my book Security Engineering. The
book also has a chapter on terror which
discusses social-scale effects. There are also survey papers: here is the latest, while an older
survey is also available as a video
with slides.
- Privacy
and human behavior in the age of information by Alessandro Acquisti, Laura
Brandimarte and George Loewenstein summarises empirical research on privacy
behaviour. People have difficulty evaluating the privacy consequences of online
actions; privacy can harm as well as help; and their concerns are highly
context-dependent.
- Economics,
Psychology and Sociology of Security by Andrew Odlyzko discusses how
cultural factors undermine the formal assumptions underlying many security
systems, and gives some insights from evolutionary psychology; for example, we
have specialised neural circuits to detect cheating in social situations.
- Privacy
in Electronic Commerce and the Economics of Immediate Gratification is one
of the seminal papers: in 2004 its author, Alessandro Acquisti, first applied
behavioral-economics ideas such as risk aversion, optimism bias and hyperbolic
discounting to understand privacy (and security) behaviour. It started to
explain the privacy paradox by showing that we cannot expect people to behave
fully rationally when confronted with typical privacy choices. (For more on
this topic, see here.)
- Human Behaviour
and Deception Detection, by Mark Frank and colleagues, surveys the state of
the art in deception detection. Most people are poor at detecting lies as we
get little practice; a small minority are gifted at it. (For more on this
topic, see here.)
- If only gay sex
caused global warming by Daniel Gilbert is a psychologist's analysis of
why we overreact to threats associated with moral norms, such as terrorism,
while discounting morally-neutral threats like environmental damage. (For more
on this topic, see here.)
- Michelle Baddeley's Information
Security: Lessons from Behavioural Economics is a survey of results from
behavioural economics of potential interst or use to security engineers,
covering risk, unertainty, bounded rationality, cognitive limits, cognitive bias
and heuristics, time inconsistency, learning, social influence, the role of
emotions and the implications for policy.
- Why
Johnny can't encrypt: A usability evaluation of PGP 5.0 was the seminal
paper that kicked off the study of security usability. Alma Whitten and Doug
Tygar showed that the then most popular encryption product was essentially
unusable, even by computer-savvy people: essentially everyone made significant
mistakes when driving it. (For more on this topic, see here.)
Deception
- To
freeze or not to freeze, by Sophie van der Zee and colleagues, shows how you
may be able to build a better lie detector by analysing body motion (Blog post Guardian Mail). A companion paper, Mining
Bodily Cues to Deception, analyses the signals that can be extracted
from different limb movements. Both papers appeared at HICSS 2015.
- The
Psychology of Scams – Provoking and Committing Errors of Judgment, by
Stephen Lea and colleagues, is an encyclopaedic study of how people fall for
online and other scams, conducted for the UK Office of Fair Trading. The
authors give a thorough literature review, describe a large number of scams,
and then present four original studies of their own. Scams involve many of the
other techniques used by legitimate marketers; people who are poor at
regulating their emotions, and perhaps are socially isolated, may be
disproportionately vulnerable, but some of them can be educated to resist
scammers.
- Understanding
scam victims: seven principles for systems security by Frank Stajano and
Paul Wilson examines a variety of scams and cons that were documented in the
BBC TV programme "The Real Hustle", extracts the principles on which the scams
were based, and applies them to system security.
- Bill Von Hippell and Robert Trivers' The
evolution and psychology of self-deception argues that we evolved
self-deception the better to deceive others. Liars who can make themselves
believe their lies can lie fluently, and minimise retribution if discovered.
The mechanisms generally involve favouring welcome over unwelcome information,
via biased strategies for search, memory and interpretation. Distinguishing
self-deceptive from other biases is a hard and interesting problem; people
can deceive themselves at both conscious and unconscious levels. There are some
limits, though: people who are good at self-deception may be more susceptible to
manipulation by others, as when abused or exploited people prefer to believe
that their sufferings are their own fault.
- Limited capacity
to lie: Cognitive load interferes with being dishonest by Anna van ’t Veer,
Mariëlle Stel and Ilja van Beest report an experiment in which subjects were
more likely to lie to get a bigger payout if they were under low cognitive load.
- Investigative
interviewing and the detection of deception, by Mark Frank and colleagues,
extends their theoretical work (above) to practical
advice, describing interviewing training they have developed for police officers
and members of the intelligence community.
- Jeff
Hancock's On Lying
and Being Lied To: A Linguistic Analysis of Deception in Computer-Mediated
Communication finds a number of statistical differences between lies and
truths told in text-based communication. His
Separating Fact
From Fiction: An Examination of Deceptive Self-Presentation in Online Dating
Profiles shows that men lied more about their height, and women lied more
about their weight; the latter class of fibs seemed intentional rather than
self-deception, and were balanced against social constraints such as the
antitipcation of future interaction.
- David Larcker and Anastasia Zakolyukina's Detecting
Deceptive Discussions in Conference Calls apply deception research to spot
untruths by corporate CEOs and CFOs in calls with analysts, where ground truth
is taken from subsequent restatements of earnings. Their techniques work
significantly better than random (see also comment in The Economist).
- Fred Cohen has
a summary
of deception research; this explores psychological phemonena relevant to
information system applications from fraud to warfare, and compares deception
at different levels between humans, computers and organisations. This may give
a framework for thinking about deceptions that combine human, computer and
organisational aspects.
- Less than human:
self-deception in the imagining of others by David Livingstone Smith makes
the point that self-deception is important in war; many people find it
difficult to kill unless they can persuade themselves that the enemy is not
fully human but somehow a predator or vermin.
- Social
Phishing, by Tom Jagatic and others, reports how the percentage of students
who responded to a test phishing email was increased from 16% to 72% by
including relevant social information about the target (for example by making
the email appear to come from a friend of the target, identified via a social
networking site).
- Stumbling Into
Bad Behavior by Max Bazerman and Ann Tenbrunsel explores the ease with
which we indulge in self-deception when it's in our interests to do so. It
doesn't necessarily help by getting people to disclose potential conflicts as
this may just make them feel absolved while making victims more trusting.
- Fred Schauer and Dick Zeckhauser's Paltering
explores untruths that stop somewhat short of a full lie – the many
half-truths, fudges, exaggerations and distortions that oil social intercourse
and are the very stuff of politics.
Security and Usability
- The
Memorability and Security of Passwords – Some Empirical Results by
Jeff Yan and others kicked off the empirical study of what sort of advice we
should give users to stop them choosing weak passwords.
- Users
Are Not The Enemy, by Anne Adams and Angela Sasse, is another classic; it
explores why users break security rules, the costs of poor usability, and what
sort of system, contextual, and work-practice changes may help: for example,
they advocate shared passwords for shared work. Her later paper The True Cost of
Unusable Password Policies is a deeper ethnographic study of the effects in
organisations of policies requiring frequent password change; this gets in the
way of productivity, and if strictly enforced leads most people to write their
passwords down (a flaw that can be fixed if you allow small changes to old
passwords).
- A Series of
Book Chapters by Peter Gutmann may constitute the most authoritative guide
to actual security usability faults in popular systems. They first give a
lot of examples of good and (especially) bad practice, then go on to discuss
underlying cognitive biases that make usability engineering hard, then to
present some guidelines for designing security interfaces. Then they discuss
how warnings and other dialogues can be made meaningful to the user, and
finally how security usability can be tested.
- It's
no secret: Measuring the security and reliability of authentication via
`secret' questions, by Stuart Schechter, Bernheim Brush and Serge Edelman,
studies the personal questions commonly used as fallback authentication for
users who forget their passwords. This methodology doesn't work very well at
all; participants forget a fifth of their own answers within six months, while
their answers can be guessed by a sixth of their acquaintances with whom they
were unwilling to share passwords.
- Jean Camp's Experimental Evaluation of
Expert and Non-expert Computer Users' Mental Models of Security Risks
explores empirically the differences between the mental models of information
security risk used by experts and non-experts.
- A
Framework for Reasoning About the Human in the Loop by Lorrie Cranor
proposes a framework to help designers understand human limitations, so that
people can be used as dependable components in systems. It is based on research
on warnings and essentially walks the designer through issues such as
attention, knowledge, comprehension and beliefs, highlighting those areas that
may need testing before the system is designed or training afterwards.
- The Emperor's New
Security Indicators, by Stuart Schechter, Rachna Dhamija, Andy Ozment and
Ian Fischer, presents an empirical study of the security indicators on which
banks and e-commerce websites rely to inform customers that all's well (or
not). These basically don't work well or in some cases at all.
- Toward a
broader view of security protocols, by Matt Blaze, advocates treating
security protocols as phenomena of human interaction and gives a number of
examples of protocols that have evolved over time to give the participants
reciprocal benefits.
- My paper A
birthday present every eleven wallets? is the first proper study of the
security of customer-selected bank PINs, and documents all sorts of bad stuff
(blog, press, blog). Another, Social
Authentication – harder than it looks, shows how Facebook's soocial
captcha system is vulnerable to guessing by friends and to face recognition
software (blog). A third, It's the Anthropology,
Stupid! discusses how we might put context and emotion back in security
decisions.
- Rick Wash's Folk
Models of Home Computer Security reports ethnographic research of how
people generally model computer security. People think of threats as
‘viruses’ which could be buggy software, or mischievous, or
designed to support crime; or ‘hackers’ who may be seen as graffiti
artists or burglars or as professionals who target big fish. The kinds of
security advice they're likely to follow depend on their main mental model;
they will disregard advice they believe is irrelevant, and we should design
systems with this in mind.
- Finally, Lorrie Cranor's
course notes on usable privacy and security give a fairly thorough canter
through the security usability literature. They include most of the above
papers and much more besides.
See also Alma Whitten's HCISec
bibliography.
Social Attitudes to Risk
- Taken Out of Context
– American Teen Sociality in Networked Publics, which is danah boyd's
thesis, is an ethnographic study of how American teenagers actually deal with
risk and information security. Passwords are often shared as a token of
intimacy when young people date; some lies (for example, about age) are
acceptable while others (about birthdays) are not. The overall picture is of
real usage being very different from websites' contract terms, and from the
mental models used by security engineers.
- The Myth of
the Superuser: Fear, Risk and Harm Online by Paul Ohm illuminates the
anthropology of computer security fearmongering. The media consistently
exaggerate the capabilities of computer experts in order to create bogeymen;
the resulting myth is nurtured by many who find it convenient, but it imposes
costs from poor systems design to over-broad laws.
- Reacting
to Terrorism: Probabilities, Consequences, and the Persistence of Fear by
John Mueller studies ways of measuring the actual (rather limited) harm that
terrorism does, versus the exaggerates fears that it provokes. Terrorism does
not cause panic directly (actual attacks lead to mutual aid); the problem is
its secondary effect in causing risk-averse behaviour such as driving instead
of flying; stress-related illnesses; and a political climate in which temperate
behaviour is harder. Dramatic first impressions do make a big difference.
- Terror,
Security, and Money: Balancing the Risks, Benefits, and Costs of Homeland
Security by John Mueller and Mark Stewart documents the cost of overreaction
to terrorism since 9/11, as in excess of a trillion dollars: $360 billion more
for homeland security, $110 billion more for intelligence, and private-sector
expenditures up by over $100 billion. These now exceed spending on all crime.
Yet the terrorism insurance premium on a $303m building is under $10,000 a year.
- A
Policy Maker's Dilemma: Preventing Terrorism or Preventing Blame by Peter
McGraw, Alexander Todorov, and Howard Kunreuther explored how politicians are
motivated to try to avert the most upsetting terrorist attacks rather than the
most likely ones, as voters' outcome biases drive political assessments more or
less regardless of a priori likelihood.
- In Cars, Cholera and
Cows: The Management of Risk and Uncertainty John Adams explores why
organisations tend to be more risk-averse than rational economic considerations
would dictate. One of the mechanisms is adverse selection: the people who end
up in risk management jobs tend to be more risk-averse than average.
- Loving the Cyber Bomb? The Dangers of Threat Inflation
in Cybersecurity Policy by Jerry Brito and Tate Watkins documents the
threat inflation techniques used by the cyber-security industrial complex to
peddle its products.
- Counterterrorism Since
9/11 analyses what's worked and what's not; its authors Nick Adams, Ted
Nordhaus and Michael Schellenberger conclude that the gains came from good
old-fashioned police work, not from the controversial measures such as data
mining, ethnic profiling and torture.
Behavioral Economics of Security
- The
Economics of Privacy systematizes the last ten year's research into the
behavioural economics of privacy. Defining privacy is hard; it can harm as well
as help; and consumers have great difficulty evaluating the privacy consequences
of their actions online.
- Privacy,
Economics and Price Discrimination describes the privacy paradox: Why is
privacy being eroded so rapidly, despite many people saying they care about it?
Andrew Odlyzko's analysis puts much of the blame on differential pricing.
Technology is increasing both the incentives and the opportunities for this.
- The
Best of Strangers: Context Dependent Willingness to Divulge Personal
Information by Leslie John, Alessandro Acquisti and George Loewenstein
describes experiments that show how context-dependent our privacy preferences
are. Students were asked to divulge sensitive personal information in neutral
setting, or in one that provided privacy reassurances: they divulged less there
as privacy had become salient. Others were asked via a frivolous website and
divulged much more.
- In Social
Networks, Personalized Advertising, and Privacy Controls, Catherine Tucker
studied the May 2010 privacy policy change on Facebook using an advertising
field test run by a nonprofit, and found that the click-through rate for
personalised advertising more than doubled after privacy policy was tightened
— while generic ads wee unaffected. The policy change affected only
consumer perceptions, not reality, as Facebook considers even personalised
advertising to be “anonymous” and thus outside its privacy
framework. She concludes that giving users the appearance of more control can
make ads more effective.
- Misplaced Confidences: Privacy and the Control Paradox by Laura Brandimarte,
Alessandro Acquisti and George Loewenstein, explored the control paradox –
the fact that we're more willing to reveal sensitive information than have the
same information revealed by others. Lower perceived control triggers lower
willingness to reveal; the effect is particularly strong for privacy-intrusive
questions, and for publication rather than sharing, access or other use.
- What
Can Behavioral Economics Teach Us About Privacy?, by Alessandro Acquisti
and Jens Grossklags, explores how research at the boundary between psychology
and economics can cast light on the privacy paradox. Framing, self-serving bias
and other effects could all contribute. Their
paper When 25 Cents is
too much: An Experiment on Willingness-To-Sell and Willingness-To-Protect
Personal Information provides a nice exposition of the difference between
willingness to pay and willingness to accept, with the latter usually being
much higher, and discuss how this can also be applied.
- A prospect theory
approach to security by Vilhelm Verendel shows how one of the oldest and
best-established tools of behavioural economics, prospect theory, can go quite
some way in explaining why security metrics are hard to do.
- Who Signed Up for the
Do-Not-Call List?, by Hal Varian, Fredrik Wallenberg and Glenn Woroch,
analyses the FCC's telephone-sales blacklist by district. Privacy means
different things to different population groups, but this raises further
questions. For example, educated people are more likely to sign up, as one
would expect: but is that because rich households get more calls, because they
value their time more, or because they understand the risks better? In Financial Privacy for
Free?, Alessandro Acquisti and Bin Zhang apply a similar analysis to credit
reporting.
- Social Interaction,
Observational Learning, and Privacy: the "Do Not Call" Registry offers
further analysis of who subscribed to the FCC's telephone-sales blacklist. Its
authors Khim Yong Goh, Kai-Lung Hui and Ivan Png found that the clustering of
people who opted out was more due to social interaction than to local news
media; that the extent of observational learning decreased with social
heterogeneity; and that people who opted out were mostly interested in escaping
telemarketers.
- The Value of
Online Information Privacy: An Empirical Investigation by Il-Horn Hann,
Kai-Lung Hui, Tom Lee and Ivan Png reports testing privacy preferences of grad
students in both the USA and Singapore. Most students were concerned to prevent
unauthorised secondary use of shopping data ("privacy guardians"); there were
minorities of "privacy sellers" and "convenience seekers" willing to trade
privacy for cash or convenience respectively.
- Privacy,
Trust and Self-Disclosure Online, by Adam Joinson and others, explores the
link between stated and revealed privacy preferences. They performed two
studies of the dispositional and situational aspects of online trust and
privacy in an attempt to establish whether trust and privacy are substitutes.
It turned out that user actions were governed more by situational than
dispositional factors.
- Sandra Petronio's Regulating the
Privacy of Confidentiality develops a theory of how people manage privacy as
a continuing process of social negotiation of inclusion or exclusion and of
using co-ownership of information as a tool to achieve various social goals.
The sharing rules can end up being as complex and dynamic as the underlying
relationships that they express. reflect and influence. The models in common use
in the infosec community aren't anything like rich enough; worse, automation
can pre-empt the negotiation process that's how we naturally behave around the
sharing of sensitive information. This has particular impact for health
privacy.
- On the Economics
of Anonymity by Alessandro Acquisti, Roger Dingledine and Paul Syverson
studies why anonymity systems are hard to sell, and points out some of their
novel aspects. For example, honest players want some level of free-riding, in
order to provide cover traffic. So equilibria can also be novel, and the ways
in which they break down can be complex. We also have to consider a wider range
of principals – dishonest, lazy, strategic, sensitive, and myopic –
than in most of the markets that economists try to model.
- The Effect of
Online Privacy Information on Purchasing Behavior: An Experimental Study by
Janice Tsai, Serge Egelman, Lorrie Cranor and Alessandro Acquisti shows that by
making information about website privacy policies more accessible and salient
it is possible to get shoppers to pay more attention to it and even to pay a
premium for privacy.
- Conformity or
Diversity: Social Implications of Transparency in Personal Data Processing
by Rainer Boehme studies whether making information widely available about the
bases on which decisions are taken about individuals will lead to more
conformity (because, in the absence of information asymmetries and strategic
interaction with others, the optimal behaviour becomes mainstream) or diversity
(as in the absence of transparency, individuals are herded together by
uncertainty and fear). He presents a model of how preferences and signaling
behaviour might interact.
- Economic
Methods and Decision Making by Security Professionals by Simon Shiu reports
an experiment which found that if a security engineering problem was framed in
economic terms, professionals gave the same advice; justified their answers with
better analysis; but denied this, claiming they understood such trade-offs
anyway.
- In Does
Deterrence Work in Reducing Information Security Policy Abuse by Employees?
Qing Hu, Zhengchuan Xu, Tamara Dinev and Hong Ling challenge the effectiveness
of deterrence in preventing insider fraud. In a survey of 207 staff at five
Chinese companies, they found that when employees decide whether to abuse
company computer systems for private gain, the benefits dominate the risks in
the decision-making process. Deterrence is thus fairly weak; the employee's
moral values and level of self-control are more important. This is consistent
with conventional criminological studies which show that actual criminals think
mostly of gains and little of negative consequences. Rather than relying on
deterrence, employers should screen staff and lower the perceived benefits of
violations.
- Why we can't
be bothered to read privacy policies – models of privacy economics as a
lemons market, by Tony Vila, Rachel Greenstadt, and David Molnar, examines
why many consumers fail to think of future price discrimination when giving
information to merchants.
See also Alessandro
Acquisti's privacy economics page.
Miscellaneous Papers
- The
Social Function of Intellect by Nick Humphrey was the paper that kicked off
the "Machiavellian Brain hypothesis" – the idea that we became smarter
than other monkeys not so we could make better tools, but so we could use other
monkeys as tools. As group sizes increased, so did the complexity of the
"social chess" that we play. Later versions of this theory can be summarised
as: monekeys who were better at deception, or detecting deception in others,
left more descendants.
- Reason
and Rationality, by philosophers Richard Samuels, Stephen Stich and Luc
Faucher, discusses the tensions between the heuristics-and-biases tradition and
modern evolutionary psychology, with the former being more pessimistic and the
latter being more optimistic about the rationality of the average man or woman.
- The
nature of collective resilience: Survivor reactions to the 2005 London
bombings, by John Drury, Chris Cocking and Steve Reicher, analyses bomb
survivors'accounts, and finds more reports of calm than panic, with the latter
tending to arise in reports by commentators who were not there, or in victims'
reports of feelings of fear, rather than in witnesses' descriptions of crowd
behaviour. Many of the victims felt a sense of solidarity and helped each
other; the authors hypothesise that a sense of a common fate drove people to be
more altruistic than normal.
- Intra-group
Regulation of Violence: Bystanders and the (De)-escalation of Violence, by
Mark Levene, Rachel Best and Paul Taylor, reports a survey of 42 incidents of
night-time violence, taken from UK CCTV camera records. Bystanders calm down
fights more often than egging on the combatants; and, contrary to previous
beliefs, this bystander effect actually increases as thr group size does.
- Paradigm
Shifts in Security Strategy by Dominic Johnson and Elizabeth Madin explores
why society is predisposed to accept the status quo until something goes
wrong. Reasons range from the discounting of hypothetical dangers through
status-quo bias, organisational inertia and the entrenched policy preferences
of dominant leaders to the lack of incentive in electoral politics for
disruptive precautions against unlikely threats. The upshot is that radical
changes often require a disaster. But democratic states with innovative
cultures are more likely to adapt than weak states are, and they might be made
more adaptable still by measures such as term limits, campaign finance limits,
freedom of information, and ensuring that decision makers talk regularly to
front-line staff.
- From
Spaces to Places: Emerging Contexts in Mobile Privacy by Clara Mancini
and others studied the privacy behaviour of mobile technology users; the paper
reports a number of different types of boundaries, based on such factors as
personal policy, inside knowledge, etiquette, proximity and aggregation, that
people used in different ways to regulate interaction.
- The
Compliance Budget: Managing Security Behaviour in Organisations, by Adam
Beautement, Angela Sasse and Mike Wonham, argues on the basis of a series of
interviews with corporate security manaagers that both people and groups
within organisations are only prepared to put up with so much regulation, and
in consequence managers should aim to make the best possible use of this
"compliance budget" rather than just issuing one instruction after another.
- In Real
Name Verification Law on the Internet: a Poison or Cure for Privacy?, Daegon
Cho investigates whether the South Korean government's real-name verification
law chilled discussion online. Since 2007 the 35 websites with over 300,000
users have restricted posting to users who register their ID numbers; people
can still use pseudonyms but their real names are known to authority. The
author crawled over 2 million postings, measuring participation, swear words
and antinormative expressions before and after implementation. Participation in
political discussion is not associated with the law change; but swear word use
and antinormative sentiments decreased slightly, especially among heavy users.
- Using Social
Psychology to Implement Security Policies, by Mich Kabay, discusses how to
get people to pay attention to security policies and the importance of the
social schemata by which people frame reality and judge behaviour. These are
particularly important when the desired behaviour violates social norms such as
holding doors open for people. A number of strategies for changing expectations
and norms are discussed.
- Finally, if you're interested in the dark side, The
Manipulation of Human Behavior by Albert Biderman and Herb Zimmer reports
experiments on interrogation carried out after the Korean War with US
Government funding. It's also known as the Torturer's Bible, and describes the
relative effectiveness of sensory deprivation, drugs, hypnosis, social pressure
and so on when interrogating and brainwashing prisoners.
Conferences
The Security and Human Behaviour workshop brings security engineers together with psychologists, behavioral economists and others. See
Decepticon is a conference on deception we organised
in August 2015.
It brought together people interested in deception, whose publications used to
be scattered between APLS, iIIRG, SARMAC, and EAPL conferences, as well as some technical and multidisciplinary events. (See also the forthcoming special issue of Cognitive Science.) Decepticon followed an earlier workshop on deception at Oxford in 2014. The second edition of Decepticon was in 2017; later events are documented
here, and the 2023 Decepticon
will take place online on December 7-8.
The Symposium On Usable Privacy and Security (SOUPS) is the
workshop for research on the usability of security systems. It has been running
since 2005; here are the programs (with links to the papers) for 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022 and 2023.
The Workshop on the Economics of Information Security (WEIS)
has some relevant papers; its focus is the interface between security and
economics. Here are the programs (with links to the papers) for 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022 and 2023.
Community – People Interested in
Security Psychology
Books
- My book Security
Engineering might be a good introduction for social scientists to secure
systems engineering. It covers not just technologies such as crypto and
firewalls, but a number of specific applications from banking to burglar
alarms. It brings out the fact that most systems don't fail because the
mechanisms are weak, but because they're used wrong.
- Beyond
Fear by Bruce Schneier discusses how we're encouraged to think about
security by law enforcement agencies, businesses and governments, and how we
can become more discerning consumers and citizens by understanding the agendas
of the various players and the means they use to influence us.
- I've described The
Art of Deception by Kevin Mitnick as "the most disturbing security book
ever". Mitnick was jailed after a hacking spree based mostly on social
engineering: he became a master at telling lies on the telephone that would
cause people to give him passwords or otherwise open systems to attack. This
book tells how he did it.
- The
Human Contibution by James Reason discusses human factors in the design of
safety-critical systems. The predictable varieties of error are rooted in the
very nature of cognition. Safety engineers have been studying psychology for
much longer than security engineers; we have a lot to to learn from them! His
earlier book, Human
Error, is also a classsic, having been a key publication in the emergence of
safety usability as a discipline.
- Darwinian
Security by Rafe Sagarin and Terry Taylor arose out of a research project
that brought together life scientists from many disciplines to review the
attack and defence strategies used by living organisms in the struggle for
survival. (See also the book website and video.)
- John Mueller, Overblown:
How Politicians and the Terrorism Industry Inflate National Security Threats,
and Why We Believe Them analyses the psychological and social factors
exploited by the fear industry, and also explores its political history. It
turns out that presidents who downplayed terrorist attacks (such as Eisenhower,
Kennedy, Nixon, Reagan and the elder Bush) did better than the presidents who
played them up (such as Carter and the younger Bush).
- Diego Gambetta's Trust
— Making and breaking cooperative relations is a seminal collection
of essays on why we have to study the psychological and underlying biological
bases for trust behaviour; economic explanations based on reciprocity are not
enough.
- Stanley Milgram's Obedience
to Authority: An Experimental View describes his classic experiment in
which he found that most subjects were prepared to administer painful and
dangerous electric shocks to a "student" when ordered to by an experimenter in
a position of ostensible authority.
- Philip Zimbardo's The
Lucifer Effect: Understanding How Good People Turn Evil describes the
infamour Stanford prisoner experiment which had to be stopped after students
playing the role of pricon guards started brutalising students playing the role
of inmates. You didn't need abuse of authority to turn good people bad: social
roles could be enough.
- Security and Usability: Designing Secure Systems that People Can Use, edited by
Lorrie Cranor and Simson Garfinkel, is a compendium of the early research papers
on security usability. With over 750 pages and 30 contributors, it is basic
reading for any serious researcher.
- Richard Byrne and Andy Whiten edited two major volumes of papers on the
Machiavellian brain
hypothesis: Machiavellian
Intelligence – Social Expertise and the Evolution of Intellect in
Monkeys, Apes and Humans
and Machiavellian
Intelligence II – Extensions and Evaluations which cover the
development of the theory that we evolved our intelligence in order to become
better at deception, and at detecting deception in others.
- Steven LeBlanc and Katherine Register's Constant
Battles: The Myth of the Peaceful, Noble Savage debuked the idea that we
used to live in harmony before technology came along. In fact, at all times in
prehistory and in history until recently, perhaps a quarter of men and boys
dies as a result of homicide. Warfare was the default means of population
control.
- Why we Lie by David
Livingstone Smith provides a detailed discussion of a simple, profound truth:
to be good at deceiving others, we have to deceive ourselves. Self-deception is
at the heart of many social phenomena.
- Risk
by John Adams is the classic study of why people and organisations are
sometimes more risk-averse than would seem rational, and sometimes more
risk-loving. For example, mandatory seat-belt laws did not reduce road traffic
casualties overall, but merely shifted them from vehicle occupants to
pedestrians and cyclists. Adams explains this by a `risk thermostat': people
compensate for an increased feeling of safety by driving faster. In general,
behaviour is governed by the probable costs and benefits of possible actions as
perceived through filters formed from experience and culture.
- Heuristics
and Biases: The Psychology of Intuitive Judgment, edited by Thomas
Gilovich, Dale Griffin and Danny Kahneman, is a collection of great papers on
behavioral economics – a subject that was founded by the third editor and
that lies at the boundary between psychology and economics. It studies how the
various heuristics and biases that underlie intuitive thinking make us prone to
irrational behaviour and various kinds of decision errors. It updates a 1982
reference, Judgment
under Uncertainty: Heuristics and Biases which brought together the seminal
papers in the field.
- Gerd Gigerenzer's Gut
Feelings: The Intelligence of the Unconscious describes how simple rules of
thumb underlie much of our subconscious mental processing and account for
decisions that we make instinctively or intuitively. These rules are
surprisingly robust in a wide range of applications, as they are founded in
evolutionary pressures on organisms to choose adaaptive courses of action in the
face of incomplete information.
- Robert Cialdini's Influence:
Science and Practice is the standard reference on the psychological science
behind marketing. Cialdini worked for several years undercover as a salesman,
fundraiser and advertiser; he distilled the experience of this fieldwork into a
study of the techniques of persuasion including reciprocation, consistency,
social proof, liking, authority, and scarcity. The book is basically a guide to
pushing people's buttons.
Other Resources
Here are some suggestions for further reading: