An Imminent Threat from Artificial Intelligence

i’m sure

that by now you’ve heard a wealth of

op-eds and talking heads herald the rise

of

artificial intelligence and along with

that you’ve probably heard

a cascade of risks that we’re going to

face

risks from ai becoming

some nightmarish immortal dictator that

we’ll never escape

from to philosophical questions

surrounding agi and super intelligence

to the end of work as a product of

automation

and fully autonomous weaponry the media

has

tended to focus on these mid to

long-term threats while in my view

neglecting

an extremely imminent if not already in

process

risk the risk that i’m concerned with

is one affecting our well-being our

mental health

our beliefs about the world in this talk

what i want to do is i want to

get us all on a common ground

concerning what we mean when we say

artificial intelligence

where we’re interacting with artificial

intelligence today

and the kinds of risks that arise from

that interaction

so to begin with what is artificial

intelligence

well this contemporary wave of progress

in the field has really been fueled by

a subfield of a subfield called deep

learning and neural networks

a neural network is just a program

consisting of a set of neurons

which are wired together at one end of

the network will feed in inputs

at the other end of the network we’ll

receive outputs

so on that input side we can feed in any

sort of data

for instance we can feed in pictures and

then we can ask the question is a baby

in this image or is it not

in the example on the screen you can see

that the network’s making a mistake

it’s claiming that there’s no baby in an

image which clearly contains a baby

so that brings us to the second point of

using neural networks

teaching them training them so what we

do is we simply let the neural network

know

you’re making a mistake and then we step

back we let the neural network

reconfigure itself in some way in the

example on the screen you’ll see

removing connections between the neurons

and as a product of this change

the correction occurs the neural network

is now expressing the correct function

so there’s this two-step procedure you

specify

some network some objective rather you

ask it to do something

and then you step back and you let the

neural network

come to its own solution its own

solution strategy to solve the problem

you posed okay so that’s a bit about how

neural networks work where are you

interacting with them

today so there’s broadly two categories

which you can kind of

lump your interactions into the first

are these companies that want you to

buy some product the second are the

companies

that collaborate with the first category

to

keep you on platform and serve you ads

i’m going to focus on the second of

these categories

these social medias these content

delivery platforms

this is where we are we spend an immense

amount of our time

we spend hours of our day interacting

with neural networks

so how do neural networks play into

these platforms well

on something like youtube the primary

method

of navigating the site is this sidebar

which contains recommendations for the

next video you should watch

this is completely determined by a

neural network

on a platform like twitter your home

feed this aggregation point of all the

accounts that you follow

is being filtered and re-ranked

according to neural networks

similarly on instagram those

posts that you’re presented with the

order in which they arrive is decided by

a neural network

okay so this is where we’re kind of

interacting with these models

what’s the risk well we know that the

content you consume has a massive impact

on your well-being

it affects your belief about the world

your mental health

and there’s this famous now infamous

study

conducted by researchers at facebook and

cornell

which seeks to address the extent to

which emotion contagion

exists within digital social networks

so emotion contagion it’s this idea that

when we’re presented with

other humans expressing an emotion we

ourselves will begin to

experience that emotion interestingly

you don’t necessarily have the ability

to recognize

that emotion is external to you you you

believe that it’s authentically your own

and so what these researchers did is

without informing users

they manipulated the news feeds of seven

hundred thousand accounts

the way they manipulated these news

feeds was to bias them towards more

positive or more negative content

and then they would record how those

manipulated users

own posts changed in sentiment

what they found was that there was an

overwhelming

positive correlation with that

manipulation

so if i go and i serve you a bunch of

negative content

your own expressions your own

communications

will become negative similarly for

positive content

so i mentioned that this study was

infamous and it’s

infamous on the level of there are some

strong questions about

the ethics of the conduct of the study

and there’s also some questions about

facebook’s chosen conclusions to draw

from the results

but the work stands as an extremely

compelling

piece of evidence for the hypothesis

that

simply by manipulating the content that

you consume

you can have immense impact on the

well-being the mental states the beliefs

of your downstream users okay so i hope

that

the pieces are starting to fall into

place for

how this current setup this current

framework that we exist in

has risk the content that we consume has

massive impact on our well-being and yet

we’ve

outsourced that role to ai

to neural networks okay so let’s take a

step back

and let’s think about how we train

neural networks

so again we specify some objective we

ask the network to do something

and then we let the algorithm arrive at

its own solution

well what’s the objective that we’re

specifying for these content serving

networks

they broadly fall into two categories

the first

is engagement engagement is the

probability that

you’ll interact with a piece of content

i serve you maybe you’ll like it you’ll

comment on it you’ll reshare it to your

friends

the second category is time

and so how can i serve you content in

order to keep you

on platform which will give me the

company

more opportunities to serve you ads

which means more opportunities

to make money so

i want to talk about this second

category time

this seems like an immensely risky

choice of objective to optimize

humans have this natural pathology of

addiction and we’re

explicitly optimizing for them to spend

time on one thing

and remember that neural networks

they’re going to take whatever strategy

they come up with

that’s most effective at achieving their

goals and if our goal is time

we really need to ask the question what

if

addicting enraging depressing

our users is the most effective strategy

to keep them on site engaged engrossed

with the content

i think that’s a very plausible

hypothesis and so the next question we

can ask is okay well

what sort of frameworks exist at the

moment to keep track of this risk

or even to mitigate it unfortunately

very few but there’s an extremely simple

and familiar framework that we can rely

on in order to

kind of defend against these risks

to begin with consider your total user

pool of your service

on facebook we’re talking about more

than two billion users

now separate a small baseline pool

and call that call that your baseline

pool and

this baseline pool what’s going to

happen is there’s going to be no

interaction

with your model so for these users in

this baseline set

they’re not going to have their feeds

ranked by a neural network it’s going to

be very simple straightforward

algorithms like

sequential serving okay the next thing

we can do is we can track these metrics

of well-being

on both pools and we can compare them

these metrics are being tracked in real

time

continuously so when we see these

metrics begin to

diverge when some gap appears we can

take

active measures to try to minimize that

gap

or reduce it these active measures might

look

something like reducing the impact a

model has on

users experience of the site so this

framework it’s

it’s incredibly general it could be

applied to pretty much any content

serving platform or

social media that exists today

but there’s still two core issues or

questions that i have with the setup the

first is

well what’s a metric of well-being you

know choosing metrics is an

incredibly difficult thing i really

don’t think

that computer scientists should be

trusted with coming up with

the fundamental metric of well-being

we’re the ones who came up with time in

the first place

so where i think this is going to come

from is a cross-disciplinary

collaboration between computer

scientists

and statisticians and ethicists and

philosophers

and social scientists

i think that this kind of

this issue is approachable and we can

make concrete steps towards solving it

so that’s the first issue i see with

this framework the second question i

have is one that

i don’t have an answer to and i’m going

to leave you with today

so when i was talking about taking some

active measure

in order to close that gap between the

metrics

i was talking about population level

dynamics

so we’re tracking these metrics on

aggregate

we’re tracking among all the model

facing users and among all the baseline

users

and then when those two pools shift in

some way

we’re taking an action across all users

to try to correct for that but that’s

not the only option

we could track metrics on a per user

basis

so we could watch each individual user

we could see how their metrics

begin to shift from some baseline or

from perhaps their

historical average and then we could

take a per user action

to push them in a different direction

i see a massive potential for good in

this framework

you can imagine a suicidal user if our

metrics can pick up

on entry into a depressive episode

we could take steps to try to mitigate

risk consider the fact that

suicidal humans are about 0.1 percent of

the population

if our intervention is only one percent

effective

at the scale of facebook billions of

users we’re talking about

tens of thousands of lives impacted

so massive potential for good but

consider what we have to accept if we

build frameworks for this

we’re allowing companies to track

our mental state and then actively

encouraging them to manipulate it

it’s kind of this huxley and nightmare

so i see both the potential for massive

good and

massive risk i don’t know where the

balance falls but i think that points to

another major problem there’s not a

public discussion

surrounding these issues these systems

these content serving systems they’ve

existed for

well over a decade now and yet there’s

no regulation

there’s no discussion of the types of

checks and balances we’d expect

to find in the technology of this

maturity

if there’s anything i want you to take

away from this talk

it’s a call to action to speak to your

family

your friends your teachers your

politicians about these issues

build an opinion through discourse

your opinions are sorely needed and we

the ones implementing this technology

we desperately want to hear them thank

you so much for your time

[Applause]