The security mirage Bruce Schneier

so security is two different things

right it’s a feeling and it’s a reality

and they’re different or you could feel

secure even if you’re not and you can be

secure even if you don’t feel it I mean

really we have two separate concepts

mapped onto the same word and what I

want to do in this talk is to split them

apart figuring out when they diverge and

how they converge and language is

actually a problem here there aren’t a

lot of good words for the concepts we’re

going to talk about so if you look at

security from economic terms it’s a

trade off every time you get some

security you’re always trading off

something and whether this is a personal

decision whether you gonna install a

burglar alarm in your home or a national

decision where you can invade some

foreign country you’re gonna trade off

something either money or time

convenience capabilities may be

fundamental liberties and the question I

asked when you look at it a security

anything is not whether this makes us

safer

but whether it’s worth the trade-off

you’ve heard in the past several years

the world is safer because Saddam

Hussein is not in power that might be

true but it’s not terribly relevant the

question is was it worth it and you know

you can make your own decision and then

you’ll decide whether the invasion was

worth it that’s how you think about

security in terms of the trade-off now

there’s often no right or wrong here

some of us have a burglar arm system at

home and some of us don’t handle to

depend on where we live whether we live

alone or have a family you know how much

cool stuff we have how much we’re

willing to accept the risk of theft

right in in politics also there are

different opinions a lot of times these

trade-offs are about more than just

security and I think that’s really

important now people have a natural

intuition about these trade-offs we make

them every day but last night in my

hotel room and I started double lock the

door or you and your car when you draw

here you know when we go eat lunch and

decide the foods not poison and we’ll

eat it and we make these trade-offs

again and again multiple times a day you

often don’t even notice them they’re

just part of being alive we all do it

every species does it right imagine a

rabbit in a field eating grass and the

rabbits gonna see a fox that rabbit will

make a security trade-off should I stay

or should I flee and if you think about

it the rabbits that are good at making

that trade-off will tend to live and

reproduce and the rabbits that are bad

at it will get eaten or starve so you’d

think that us as a successful species on

the planet right you me everybody would

be really good at making these

trade-offs yet it seems again and again

that we’re hopelessly bad at it right

and I think that’s a fundamentally

interesting question I’ll give you the

short answer the answer is we respond to

the feeling of security and not the

reality now most of the time that works

right most of the time feeling and

reality are the same certainly that’s

true for most of human prehistory right

we’ve developed this ability because it

makes evolutionary sense I mean one way

to think of it is that we’re highly

optimized for risk decisions that are

endemic to living in small family groups

in the East African Highlands 100,000 BC

2010 New York you know not so much now

there are several biases and risk

perception a lot of good experiments in

this and you can see certain biases that

come up again and again I’ll give you

four we tend to exaggerate spectacular

and rare risks and downplay common risks

so flying versus driving the unknown is

Recife riskier than a familiar one

example would be people fear kidnapping

by strangers when the data supports

kidnapping by relatives is much more

common this is for children

third personified risks are perceived to

be greater than anonymous risks so bin

Laden is scarier because he has a name

and the fourth is people underestimate

secure risks in situations they do

control and overestimate them in

situations they don’t control so once

you take up skydiving or smoking you

downplay the risks if a risk is thrust

upon you terrorism is a good example

you’ll overplay it because you don’t

feel like it’s in your control there are

a bunch of other of these biases these

cognitive biases that affect our risk

decisions

there’s the availability heuristic which

basically means we estimate the

probability of something by how easy it

is to bring instances of it to mind so

you guys imagine how that works if you

hear a lot about Tiger attacks let’s be

a lot of tigers around you don’t hear

about lion attacks aren’t a lot of lions

around this works until you invent

newspapers because what newspapers do is

they repeat again and again rare risks

and I tell people if it’s in the news

don’t worry about it because by

definition news is something that almost

never happens right when something is so

common it’s no longer news if a car

crashes domestic violence those are the

risks you worry about we’re also a

species of storytellers we respond to

stories more than data and there’s some

basic and numeracy going on I mean the

joke one two three many is kind of right

we’re really good at small numbers one

mango two mangoes three mangoes 10,000

mangoes 1000 mangoes it stole more

mangoes you can eat before they rot so

1/2 1/4 1/5 we’re good at that one in a

million one in a billion they’re both

almost never so we have trouble with the

risks that aren’t very common and what

these cognitive biases do is they act as

filters between us and reality right and

the result is that feeling and reality

get out of whack they get different now

you either have you know feeling you

feel more secure than you are there’s a

false sense of security or the other

away and there’s often a full sense of

insecurity I write a lot about security

theater which are products that make

people feel secure but don’t actually do

anything there’s no real word for stuff

that makes us secure but doesn’t make us

feel secure maybe it’s what the CIA

supposed to do for us so back to

economics if economics if the market

drives security and if people make

trade-offs based on the feeling of

security then the smart thing for

companies to do for the economic

incentives are to make people feel

secure right and there are two ways to

do this one you can make people actually

secure and hope they notice or two you

can make people just feel secure and

hope they don’t notice right so what

makes people notice but a couple of

things understanding right of the

security of the risks the threats the

countermeasures how they work but if you

know stuff you’re more likely to have

your feelings match reality enough real

world examples helps now we all know the

crime rate in our neighborhood because

we live there and we get a feeling about

it that basically matches reality right

security theaters exposed when it’s

obvious that it’s not working properly

okay so what makes people not notice

well a poor understanding right if you

don’t understand the risks you don’t

ascend the costs you’re likely to get

the trade-off wrong and your feeling

doesn’t match reality but not enough

examples there’s an inherent problem

with low probability events if for

example terrorism almost never happens

it’s really hard to judge the efficacy

of counter-terrorist measures

night I mean this is why you you know

you keep sacrificing virgins and while

your unicorn defenses are working just

great

there’s aren’t enough examples of

failures also feelings that are clouding

the issues right the cognitive biases

are talking about earlier fears folk

beliefs right basically an inadequate

model of reality so let me complicate

things right I have feeling and reality

I want to add a third element I want to

add model a feeling and model in our

ahead

reality is the outside world it doesn’t

change it’s real my it’s a feeling is

based on our intuition model is based on

reason that’s basically the difference

uh in a primitive and simple world

there’s really no reason for a model

maybe because this feeling is close to

reality you don’t need a model but in a

modern and complex world you need models

to understand a lot of the risks we face

you know there’s no feeling about germs

you need a model to understand them so

this model is an intelligent reputation

of reality it’s of course limited by

science by technology right we couldn’t

have a germ theory of disease before we

invented the microscope to see them it’s

limited by our cognitive biases but it

has the ability to override our feelings

where do we get these models we get them

from others we get them from religion

from culture teachers elders couple

years ago I was in South Africa on

safari the tracker I was with grew up in

Kruger National Park he had some very

complex models on how to survive and it

depends if you attacked by a lion or a

leopard or a rhino and or elephant and

one you had to run away and when you

couldn’t run away and when you had a

climate tree when you could never climb

a tree I would have you know died and in

a day all right but he was born there

and he understood how to survive right I

was born in New York City I could take

it him to New York and he would have

died in a day

right because we had different models

based on our different experiences

models can come from the media from our

elected officials right think of models

of terrorism a child kidnapping airline

safety car safety models who come from

industry and a - I’m following are our

surveillance cameras ID cards a quite a

lot of our computer security models come

from there a lot of models come from

science and health models are a great

example think of cancer of bird flu

swine flu SARS I mean all of our

feelings of security about those

diseases come from models given to us

really by science filter through the

media right so models can change right

models are not static as we become more

comfortable in our environments our

model can move closer to our feelings so

an example might be if you go back a

hundred years ago when electricity was

first becoming common there were a lot

of fears about it I mean there are

people who were afraid to push doorbells

because electricity in there and that

was dangerous right for us

we’re very faasil around electricity

which changed light bulbs without even

thinking about it

right our model of security around

electricity it’s something we were we

were born into you know it hasn’t

changed as we were growing up and and

we’re good at it right or think of the

risks I’m on the internet across

generations how your parents approach

the Internet security versus how you do

versus how are our kids will you know

models eventually fade into the

background

you know intuitive just another word for

familiar right so as your model is close

to reality and converges with feelings

you often don’t even know it’s there

so a nice example of this came from last

year and swine swine flu when swine flu

first appeared the initial news caused a

lot of overreaction

now it was it had a name which made it

scarier than the regular flu even though

is more deadly and people thought

doctors should be able to deal with it

so was there was that feeling of lack of

control and those two things made the

risk more than it was as as the novelty

wore off the months went by there was

some amount of Tolerance people got used

to it right there was no new data but

there was less fear by autumn people

thought the doctor should have solved

this already and there’s kind of a

bifurcation people had to choose between

sort of fear and and an acceptance

actually fear an indifference that kind

of choose suspicion and when the vaccine

appeared last winter there are a lot of

people a surprising number who refused

to get it it’s a nice example of how

people’s feelings of security change how

their model changes sort of wildly with

no new information with no new input

this kind of thing happens a lot but

give it one more complication and we

have feeling model reality I have a very

relativistic view of security I think it

depends on the observer and most

security decisions have a variety of

people involved and stakeholders with

specific trade-offs will try to

influence the decision and I call that

their agenda and you see agenda this is

marketing this is politics trying to

convince you to have one model versus

another trying to convince you to ignore

a model and you trust your feelings

marginalizing people with models you

don’t like maybe this is not uncommon

now example a great example is the risk

of smoking in the history the past 50

years the smoking risk shows how a model

changes it also shows how an industry

fights against the model it doesn’t like

I compare that to the second hand smoke

debate no probably about 20 years behind

think about seatbelts when I was a kid

no one wore a seat belt

nowadays no kid will let you drive if

you’re not wearing a seat belt I compare

that to the airbag debate probably about

thirty years behind all examples of

models changing what we learn is that

changing models is hard right models are

hard to dislodge if they equal your

feelings you don’t even know you have a

model and there’s another cognitive bias

I’ll call confirmation bias where we

tend to accept data that confirms our

beliefs and reject data that contradicts

our beliefs so evidence against our

model we’re likely to ignore even if

it’s compelling has to get very

compelling before I’ll pay attention new

models that extend long periods of time

are hard global warming is a great

example we’re terrible at models that

span 80 years now we can do to the next

harvest we can often do till our kids

grow up but 80 years we’re just not good

at so it’s a very hard model to accept

you know we can have both models in our

head simultaneously or that kind of you

know that kind of problem where we have

is holding both beliefs together are the

cognitive dissidence eventually the new

model replaced the old model strong

feelings can create a model by September

11th created a social security model and

a lot of people’s heads also personal

experiences with crime can do it

personal health scare health scare in

the news you’ll see is called flash mob

events by spy psychiatrists

right they can create a model

instantaneously because they’re very

emotive so in the technological world we

don’t have experience to judge models

and and we rely on others we rely on

proxies I mean this works as long as

it’s the correct others right we rely on

government Asians to tell us what

pharmaceuticals are safe I flew here

yesterday I didn’t check the air of

airplane you know I relied on some other

group to determine whether my plane was

safe to fly we’re here none of us feared

the roof is gonna collapse on us not

because we checked but because we’re

pretty sure the building codes here are

good I mean we we it’s a model we just

accept pretty much by faith and that’s

okay now what we want is people to get

familiar enough with better models right

have it reflected in their feelings to

allow them to make security trade-offs

now when use these go out of whack you

sort of have you have two options one

you can fix people’s feelings right

directly appeal to feelings it’s

manipulation but it can work the second

more honest way is to actually fix the

model right change happens slowly the

smoking debate took 40 years and that

was an easy one but some of this stuff

is hard I mean really though information

seems like our best hope and and I lied

member I said feeling or feeling model

realities in reality doesn’t change it

actually does we live in a technological

world reality changes all the time so we

might have for the first time in our

species feeling chases model model

chases reality reality is moving they

might never catch up right we don’t know

right but in the long term both feeling

and reality are important and I want to

close with two quick stories to

illustrate this 1982 I know people

remember this there was a

epidemic of tylenol poisonings the

United States it’s a horrific story

someone took a bottle Tylenol put poison

in it

close it up put it back on the shelf

someone else bought it and died this

terrified people there are a couple of

copycat attacks there wasn’t any real

risk but people were scared and this is

how the tamper-proof drug industry was

invented those chamber of caps that came

from this it’s complete security theater

you know as a homework assignment think

of ten ways to get around it give you

one a syringe right but it made people

feel better it made their feeling of

security more match to the reality

last story a few years ago a friend of

mine gave birth visited her in the

hospital turns out when a baby’s born

now they put an RFID brace on the baby

put a corresponding one on the mother

said anyone other than the mother takes

the baby out of the maternity ward alarm

goes off I said well that’s kind of neat

I wonder how rampant baby snatching is

that hospitals I go home I look it up it

basically never happens but if you think

about it if you are a hospital and you

need to take a baby away from its mother

on other room to run some tests you

better have some good security theater

or she’s gonna rip your arm off so it’s

important for us those of us who design

security will look at the security

policy or even look at public policy in

a ways that effects security right it’s

not just a reality it’s feeling in

reality what’s important is that they be

about the same it’s important that if

our feelings match reality we make

better security trade-offs thank you