The security mirage Bruce Schneier
so security is two different things
right it’s a feeling and it’s a reality
and they’re different or you could feel
secure even if you’re not and you can be
secure even if you don’t feel it I mean
really we have two separate concepts
mapped onto the same word and what I
want to do in this talk is to split them
apart figuring out when they diverge and
how they converge and language is
actually a problem here there aren’t a
lot of good words for the concepts we’re
going to talk about so if you look at
security from economic terms it’s a
trade off every time you get some
security you’re always trading off
something and whether this is a personal
decision whether you gonna install a
burglar alarm in your home or a national
decision where you can invade some
foreign country you’re gonna trade off
something either money or time
convenience capabilities may be
fundamental liberties and the question I
asked when you look at it a security
anything is not whether this makes us
safer
but whether it’s worth the trade-off
you’ve heard in the past several years
the world is safer because Saddam
Hussein is not in power that might be
true but it’s not terribly relevant the
question is was it worth it and you know
you can make your own decision and then
you’ll decide whether the invasion was
worth it that’s how you think about
security in terms of the trade-off now
there’s often no right or wrong here
some of us have a burglar arm system at
home and some of us don’t handle to
depend on where we live whether we live
alone or have a family you know how much
cool stuff we have how much we’re
willing to accept the risk of theft
right in in politics also there are
different opinions a lot of times these
trade-offs are about more than just
security and I think that’s really
important now people have a natural
intuition about these trade-offs we make
them every day but last night in my
hotel room and I started double lock the
door or you and your car when you draw
here you know when we go eat lunch and
decide the foods not poison and we’ll
eat it and we make these trade-offs
again and again multiple times a day you
often don’t even notice them they’re
just part of being alive we all do it
every species does it right imagine a
rabbit in a field eating grass and the
rabbits gonna see a fox that rabbit will
make a security trade-off should I stay
or should I flee and if you think about
it the rabbits that are good at making
that trade-off will tend to live and
reproduce and the rabbits that are bad
at it will get eaten or starve so you’d
think that us as a successful species on
the planet right you me everybody would
be really good at making these
trade-offs yet it seems again and again
that we’re hopelessly bad at it right
and I think that’s a fundamentally
interesting question I’ll give you the
short answer the answer is we respond to
the feeling of security and not the
reality now most of the time that works
right most of the time feeling and
reality are the same certainly that’s
true for most of human prehistory right
we’ve developed this ability because it
makes evolutionary sense I mean one way
to think of it is that we’re highly
optimized for risk decisions that are
endemic to living in small family groups
in the East African Highlands 100,000 BC
2010 New York you know not so much now
there are several biases and risk
perception a lot of good experiments in
this and you can see certain biases that
come up again and again I’ll give you
four we tend to exaggerate spectacular
and rare risks and downplay common risks
so flying versus driving the unknown is
Recife riskier than a familiar one
example would be people fear kidnapping
by strangers when the data supports
kidnapping by relatives is much more
common this is for children
third personified risks are perceived to
be greater than anonymous risks so bin
Laden is scarier because he has a name
and the fourth is people underestimate
secure risks in situations they do
control and overestimate them in
situations they don’t control so once
you take up skydiving or smoking you
downplay the risks if a risk is thrust
upon you terrorism is a good example
you’ll overplay it because you don’t
feel like it’s in your control there are
a bunch of other of these biases these
cognitive biases that affect our risk
decisions
there’s the availability heuristic which
basically means we estimate the
probability of something by how easy it
is to bring instances of it to mind so
you guys imagine how that works if you
hear a lot about Tiger attacks let’s be
a lot of tigers around you don’t hear
about lion attacks aren’t a lot of lions
around this works until you invent
newspapers because what newspapers do is
they repeat again and again rare risks
and I tell people if it’s in the news
don’t worry about it because by
definition news is something that almost
never happens right when something is so
common it’s no longer news if a car
crashes domestic violence those are the
risks you worry about we’re also a
species of storytellers we respond to
stories more than data and there’s some
basic and numeracy going on I mean the
joke one two three many is kind of right
we’re really good at small numbers one
mango two mangoes three mangoes 10,000
mangoes 1000 mangoes it stole more
mangoes you can eat before they rot so
1/2 1/4 1/5 we’re good at that one in a
million one in a billion they’re both
almost never so we have trouble with the
risks that aren’t very common and what
these cognitive biases do is they act as
filters between us and reality right and
the result is that feeling and reality
get out of whack they get different now
you either have you know feeling you
feel more secure than you are there’s a
false sense of security or the other
away and there’s often a full sense of
insecurity I write a lot about security
theater which are products that make
people feel secure but don’t actually do
anything there’s no real word for stuff
that makes us secure but doesn’t make us
feel secure maybe it’s what the CIA
supposed to do for us so back to
economics if economics if the market
drives security and if people make
trade-offs based on the feeling of
security then the smart thing for
companies to do for the economic
incentives are to make people feel
secure right and there are two ways to
do this one you can make people actually
secure and hope they notice or two you
can make people just feel secure and
hope they don’t notice right so what
makes people notice but a couple of
things understanding right of the
security of the risks the threats the
countermeasures how they work but if you
know stuff you’re more likely to have
your feelings match reality enough real
world examples helps now we all know the
crime rate in our neighborhood because
we live there and we get a feeling about
it that basically matches reality right
security theaters exposed when it’s
obvious that it’s not working properly
okay so what makes people not notice
well a poor understanding right if you
don’t understand the risks you don’t
ascend the costs you’re likely to get
the trade-off wrong and your feeling
doesn’t match reality but not enough
examples there’s an inherent problem
with low probability events if for
example terrorism almost never happens
it’s really hard to judge the efficacy
of counter-terrorist measures
night I mean this is why you you know
you keep sacrificing virgins and while
your unicorn defenses are working just
great
there’s aren’t enough examples of
failures also feelings that are clouding
the issues right the cognitive biases
are talking about earlier fears folk
beliefs right basically an inadequate
model of reality so let me complicate
things right I have feeling and reality
I want to add a third element I want to
add model a feeling and model in our
ahead
reality is the outside world it doesn’t
change it’s real my it’s a feeling is
based on our intuition model is based on
reason that’s basically the difference
uh in a primitive and simple world
there’s really no reason for a model
maybe because this feeling is close to
reality you don’t need a model but in a
modern and complex world you need models
to understand a lot of the risks we face
you know there’s no feeling about germs
you need a model to understand them so
this model is an intelligent reputation
of reality it’s of course limited by
science by technology right we couldn’t
have a germ theory of disease before we
invented the microscope to see them it’s
limited by our cognitive biases but it
has the ability to override our feelings
where do we get these models we get them
from others we get them from religion
from culture teachers elders couple
years ago I was in South Africa on
safari the tracker I was with grew up in
Kruger National Park he had some very
complex models on how to survive and it
depends if you attacked by a lion or a
leopard or a rhino and or elephant and
one you had to run away and when you
couldn’t run away and when you had a
climate tree when you could never climb
a tree I would have you know died and in
a day all right but he was born there
and he understood how to survive right I
was born in New York City I could take
it him to New York and he would have
died in a day
right because we had different models
based on our different experiences
models can come from the media from our
elected officials right think of models
of terrorism a child kidnapping airline
safety car safety models who come from
industry and a - I’m following are our
surveillance cameras ID cards a quite a
lot of our computer security models come
from there a lot of models come from
science and health models are a great
example think of cancer of bird flu
swine flu SARS I mean all of our
feelings of security about those
diseases come from models given to us
really by science filter through the
media right so models can change right
models are not static as we become more
comfortable in our environments our
model can move closer to our feelings so
an example might be if you go back a
hundred years ago when electricity was
first becoming common there were a lot
of fears about it I mean there are
people who were afraid to push doorbells
because electricity in there and that
was dangerous right for us
we’re very faasil around electricity
which changed light bulbs without even
thinking about it
right our model of security around
electricity it’s something we were we
were born into you know it hasn’t
changed as we were growing up and and
we’re good at it right or think of the
risks I’m on the internet across
generations how your parents approach
the Internet security versus how you do
versus how are our kids will you know
models eventually fade into the
background
you know intuitive just another word for
familiar right so as your model is close
to reality and converges with feelings
you often don’t even know it’s there
so a nice example of this came from last
year and swine swine flu when swine flu
first appeared the initial news caused a
lot of overreaction
now it was it had a name which made it
scarier than the regular flu even though
is more deadly and people thought
doctors should be able to deal with it
so was there was that feeling of lack of
control and those two things made the
risk more than it was as as the novelty
wore off the months went by there was
some amount of Tolerance people got used
to it right there was no new data but
there was less fear by autumn people
thought the doctor should have solved
this already and there’s kind of a
bifurcation people had to choose between
sort of fear and and an acceptance
actually fear an indifference that kind
of choose suspicion and when the vaccine
appeared last winter there are a lot of
people a surprising number who refused
to get it it’s a nice example of how
people’s feelings of security change how
their model changes sort of wildly with
no new information with no new input
this kind of thing happens a lot but
give it one more complication and we
have feeling model reality I have a very
relativistic view of security I think it
depends on the observer and most
security decisions have a variety of
people involved and stakeholders with
specific trade-offs will try to
influence the decision and I call that
their agenda and you see agenda this is
marketing this is politics trying to
convince you to have one model versus
another trying to convince you to ignore
a model and you trust your feelings
marginalizing people with models you
don’t like maybe this is not uncommon
now example a great example is the risk
of smoking in the history the past 50
years the smoking risk shows how a model
changes it also shows how an industry
fights against the model it doesn’t like
I compare that to the second hand smoke
debate no probably about 20 years behind
think about seatbelts when I was a kid
no one wore a seat belt
nowadays no kid will let you drive if
you’re not wearing a seat belt I compare
that to the airbag debate probably about
thirty years behind all examples of
models changing what we learn is that
changing models is hard right models are
hard to dislodge if they equal your
feelings you don’t even know you have a
model and there’s another cognitive bias
I’ll call confirmation bias where we
tend to accept data that confirms our
beliefs and reject data that contradicts
our beliefs so evidence against our
model we’re likely to ignore even if
it’s compelling has to get very
compelling before I’ll pay attention new
models that extend long periods of time
are hard global warming is a great
example we’re terrible at models that
span 80 years now we can do to the next
harvest we can often do till our kids
grow up but 80 years we’re just not good
at so it’s a very hard model to accept
you know we can have both models in our
head simultaneously or that kind of you
know that kind of problem where we have
is holding both beliefs together are the
cognitive dissidence eventually the new
model replaced the old model strong
feelings can create a model by September
11th created a social security model and
a lot of people’s heads also personal
experiences with crime can do it
personal health scare health scare in
the news you’ll see is called flash mob
events by spy psychiatrists
right they can create a model
instantaneously because they’re very
emotive so in the technological world we
don’t have experience to judge models
and and we rely on others we rely on
proxies I mean this works as long as
it’s the correct others right we rely on
government Asians to tell us what
pharmaceuticals are safe I flew here
yesterday I didn’t check the air of
airplane you know I relied on some other
group to determine whether my plane was
safe to fly we’re here none of us feared
the roof is gonna collapse on us not
because we checked but because we’re
pretty sure the building codes here are
good I mean we we it’s a model we just
accept pretty much by faith and that’s
okay now what we want is people to get
familiar enough with better models right
have it reflected in their feelings to
allow them to make security trade-offs
now when use these go out of whack you
sort of have you have two options one
you can fix people’s feelings right
directly appeal to feelings it’s
manipulation but it can work the second
more honest way is to actually fix the
model right change happens slowly the
smoking debate took 40 years and that
was an easy one but some of this stuff
is hard I mean really though information
seems like our best hope and and I lied
member I said feeling or feeling model
realities in reality doesn’t change it
actually does we live in a technological
world reality changes all the time so we
might have for the first time in our
species feeling chases model model
chases reality reality is moving they
might never catch up right we don’t know
right but in the long term both feeling
and reality are important and I want to
close with two quick stories to
illustrate this 1982 I know people
remember this there was a
epidemic of tylenol poisonings the
United States it’s a horrific story
someone took a bottle Tylenol put poison
in it
close it up put it back on the shelf
someone else bought it and died this
terrified people there are a couple of
copycat attacks there wasn’t any real
risk but people were scared and this is
how the tamper-proof drug industry was
invented those chamber of caps that came
from this it’s complete security theater
you know as a homework assignment think
of ten ways to get around it give you
one a syringe right but it made people
feel better it made their feeling of
security more match to the reality
last story a few years ago a friend of
mine gave birth visited her in the
hospital turns out when a baby’s born
now they put an RFID brace on the baby
put a corresponding one on the mother
said anyone other than the mother takes
the baby out of the maternity ward alarm
goes off I said well that’s kind of neat
I wonder how rampant baby snatching is
that hospitals I go home I look it up it
basically never happens but if you think
about it if you are a hospital and you
need to take a baby away from its mother
on other room to run some tests you
better have some good security theater
or she’s gonna rip your arm off so it’s
important for us those of us who design
security will look at the security
policy or even look at public policy in
a ways that effects security right it’s
not just a reality it’s feeling in
reality what’s important is that they be
about the same it’s important that if
our feelings match reality we make
better security trade-offs thank you