An Imminent Threat from Artificial Intelligence
i’m sure
that by now you’ve heard a wealth of
op-eds and talking heads herald the rise
of
artificial intelligence and along with
that you’ve probably heard
a cascade of risks that we’re going to
face
risks from ai becoming
some nightmarish immortal dictator that
we’ll never escape
from to philosophical questions
surrounding agi and super intelligence
to the end of work as a product of
automation
and fully autonomous weaponry the media
has
tended to focus on these mid to
long-term threats while in my view
neglecting
an extremely imminent if not already in
process
risk the risk that i’m concerned with
is one affecting our well-being our
mental health
our beliefs about the world in this talk
what i want to do is i want to
get us all on a common ground
concerning what we mean when we say
artificial intelligence
where we’re interacting with artificial
intelligence today
and the kinds of risks that arise from
that interaction
so to begin with what is artificial
intelligence
well this contemporary wave of progress
in the field has really been fueled by
a subfield of a subfield called deep
learning and neural networks
a neural network is just a program
consisting of a set of neurons
which are wired together at one end of
the network will feed in inputs
at the other end of the network we’ll
receive outputs
so on that input side we can feed in any
sort of data
for instance we can feed in pictures and
then we can ask the question is a baby
in this image or is it not
in the example on the screen you can see
that the network’s making a mistake
it’s claiming that there’s no baby in an
image which clearly contains a baby
so that brings us to the second point of
using neural networks
teaching them training them so what we
do is we simply let the neural network
know
you’re making a mistake and then we step
back we let the neural network
reconfigure itself in some way in the
example on the screen you’ll see
removing connections between the neurons
and as a product of this change
the correction occurs the neural network
is now expressing the correct function
so there’s this two-step procedure you
specify
some network some objective rather you
ask it to do something
and then you step back and you let the
neural network
come to its own solution its own
solution strategy to solve the problem
you posed okay so that’s a bit about how
neural networks work where are you
interacting with them
today so there’s broadly two categories
which you can kind of
lump your interactions into the first
are these companies that want you to
buy some product the second are the
companies
that collaborate with the first category
to
keep you on platform and serve you ads
i’m going to focus on the second of
these categories
these social medias these content
delivery platforms
this is where we are we spend an immense
amount of our time
we spend hours of our day interacting
with neural networks
so how do neural networks play into
these platforms well
on something like youtube the primary
method
of navigating the site is this sidebar
which contains recommendations for the
next video you should watch
this is completely determined by a
neural network
on a platform like twitter your home
feed this aggregation point of all the
accounts that you follow
is being filtered and re-ranked
according to neural networks
similarly on instagram those
posts that you’re presented with the
order in which they arrive is decided by
a neural network
okay so this is where we’re kind of
interacting with these models
what’s the risk well we know that the
content you consume has a massive impact
on your well-being
it affects your belief about the world
your mental health
and there’s this famous now infamous
study
conducted by researchers at facebook and
cornell
which seeks to address the extent to
which emotion contagion
exists within digital social networks
so emotion contagion it’s this idea that
when we’re presented with
other humans expressing an emotion we
ourselves will begin to
experience that emotion interestingly
you don’t necessarily have the ability
to recognize
that emotion is external to you you you
believe that it’s authentically your own
and so what these researchers did is
without informing users
they manipulated the news feeds of seven
hundred thousand accounts
the way they manipulated these news
feeds was to bias them towards more
positive or more negative content
and then they would record how those
manipulated users
own posts changed in sentiment
what they found was that there was an
overwhelming
positive correlation with that
manipulation
so if i go and i serve you a bunch of
negative content
your own expressions your own
communications
will become negative similarly for
positive content
so i mentioned that this study was
infamous and it’s
infamous on the level of there are some
strong questions about
the ethics of the conduct of the study
and there’s also some questions about
facebook’s chosen conclusions to draw
from the results
but the work stands as an extremely
compelling
piece of evidence for the hypothesis
that
simply by manipulating the content that
you consume
you can have immense impact on the
well-being the mental states the beliefs
of your downstream users okay so i hope
that
the pieces are starting to fall into
place for
how this current setup this current
framework that we exist in
has risk the content that we consume has
massive impact on our well-being and yet
we’ve
outsourced that role to ai
to neural networks okay so let’s take a
step back
and let’s think about how we train
neural networks
so again we specify some objective we
ask the network to do something
and then we let the algorithm arrive at
its own solution
well what’s the objective that we’re
specifying for these content serving
networks
they broadly fall into two categories
the first
is engagement engagement is the
probability that
you’ll interact with a piece of content
i serve you maybe you’ll like it you’ll
comment on it you’ll reshare it to your
friends
the second category is time
and so how can i serve you content in
order to keep you
on platform which will give me the
company
more opportunities to serve you ads
which means more opportunities
to make money so
i want to talk about this second
category time
this seems like an immensely risky
choice of objective to optimize
humans have this natural pathology of
addiction and we’re
explicitly optimizing for them to spend
time on one thing
and remember that neural networks
they’re going to take whatever strategy
they come up with
that’s most effective at achieving their
goals and if our goal is time
we really need to ask the question what
if
addicting enraging depressing
our users is the most effective strategy
to keep them on site engaged engrossed
with the content
i think that’s a very plausible
hypothesis and so the next question we
can ask is okay well
what sort of frameworks exist at the
moment to keep track of this risk
or even to mitigate it unfortunately
very few but there’s an extremely simple
and familiar framework that we can rely
on in order to
kind of defend against these risks
to begin with consider your total user
pool of your service
on facebook we’re talking about more
than two billion users
now separate a small baseline pool
and call that call that your baseline
pool and
this baseline pool what’s going to
happen is there’s going to be no
interaction
with your model so for these users in
this baseline set
they’re not going to have their feeds
ranked by a neural network it’s going to
be very simple straightforward
algorithms like
sequential serving okay the next thing
we can do is we can track these metrics
of well-being
on both pools and we can compare them
these metrics are being tracked in real
time
continuously so when we see these
metrics begin to
diverge when some gap appears we can
take
active measures to try to minimize that
gap
or reduce it these active measures might
look
something like reducing the impact a
model has on
users experience of the site so this
framework it’s
it’s incredibly general it could be
applied to pretty much any content
serving platform or
social media that exists today
but there’s still two core issues or
questions that i have with the setup the
first is
well what’s a metric of well-being you
know choosing metrics is an
incredibly difficult thing i really
don’t think
that computer scientists should be
trusted with coming up with
the fundamental metric of well-being
we’re the ones who came up with time in
the first place
so where i think this is going to come
from is a cross-disciplinary
collaboration between computer
scientists
and statisticians and ethicists and
philosophers
and social scientists
i think that this kind of
this issue is approachable and we can
make concrete steps towards solving it
so that’s the first issue i see with
this framework the second question i
have is one that
i don’t have an answer to and i’m going
to leave you with today
so when i was talking about taking some
active measure
in order to close that gap between the
metrics
i was talking about population level
dynamics
so we’re tracking these metrics on
aggregate
we’re tracking among all the model
facing users and among all the baseline
users
and then when those two pools shift in
some way
we’re taking an action across all users
to try to correct for that but that’s
not the only option
we could track metrics on a per user
basis
so we could watch each individual user
we could see how their metrics
begin to shift from some baseline or
from perhaps their
historical average and then we could
take a per user action
to push them in a different direction
i see a massive potential for good in
this framework
you can imagine a suicidal user if our
metrics can pick up
on entry into a depressive episode
we could take steps to try to mitigate
risk consider the fact that
suicidal humans are about 0.1 percent of
the population
if our intervention is only one percent
effective
at the scale of facebook billions of
users we’re talking about
tens of thousands of lives impacted
so massive potential for good but
consider what we have to accept if we
build frameworks for this
we’re allowing companies to track
our mental state and then actively
encouraging them to manipulate it
it’s kind of this huxley and nightmare
so i see both the potential for massive
good and
massive risk i don’t know where the
balance falls but i think that points to
another major problem there’s not a
public discussion
surrounding these issues these systems
these content serving systems they’ve
existed for
well over a decade now and yet there’s
no regulation
there’s no discussion of the types of
checks and balances we’d expect
to find in the technology of this
maturity
if there’s anything i want you to take
away from this talk
it’s a call to action to speak to your
family
your friends your teachers your
politicians about these issues
build an opinion through discourse
your opinions are sorely needed and we
the ones implementing this technology
we desperately want to hear them thank
you so much for your time
[Applause]