Augmented Intelligence

[Music]

30 years ago when i was an undergrad

ai suddenly became really popular there

was this wave of enthusiasm that swept

across the campus flipped across the

business world

it seemed like it was only a matter of

time before we were all going to be made

redundant

then aia faltered and it faded

and just went away now it’s back same

enthusiasm same promises

i wonder how well it’s going to do this

time around

how far it’s going to be able to

actually go

all aa systems have got two parts to

them the first thing an ai system has to

do is figure out where it’s at that

means it takes an input

this is data from sensors cameras and it

builds a model of the world

and based on that model of the world it

has to go ahead and decide

what kind of actions it should take and

decisions it has to make

and hopefully these are good but we look

at the

modeling portion it’s really an

important part of the ai system because

the model allows the ai system to make

predictions if the model is

poor then the predictions will be poor

and the ai system is going to end up

making

bad decisions you end up making mistakes

so here’s a couple of questions to think

about with regards to the ass system the

first one is

do a systems how do they do when they

encounter the unknown

and the second one is do ai systems have

any sense

of being lost or when they’re lost so

i’ve got two examples that help explain

and illustrate these answer these

questions the first example

will help us understand what i mean by

an ai system encountering the unknown

the second example will help us see how

easy it is for an ai system to become

lost so my first example i’m assuming

that i’m going to

build a drone delivery system in new

york city so clearly

one of the most important parts of this

is knowing for every location you can be

in new york city

how high you have to fly to make sure

you don’t run into buildings

so i’ve got a map of lower manhattan

this starts top of it is at the bottom

part of central park it goes down to the

southern tip

and i’ve got a couple of points

highlighted here just landmarks

give you some reference first one is new

york public library it’s by bryant park

the second one is the empire state

building a couple of blocks south from

the new york public library

and then a couple of miles down on the

southern tip is the one world trade

center

now this map is actually 3d data it’s a

3d rendering

of all the buildings in new york it was

done by new york city back in 2014

available on the web it’s it’s a pretty

cool set of data

i used this in order to create my

training data because that’s the first

thing i have to have in order to

build an ai system and to build a

predictive model

now i took 50 000 randomly selected

points

locations and then calculated the

heights but i focused all of these

around new york public library so you

can see that kind of concentric

almost target-like set of circles that

shows you where the data is centered

so most of the data is around the new

york public library a little bit around

the empire state building and there’s no

data down by one world trade center

and what i mean by known and unknown for

an ai system

an ai system is in known territory when

it’s got training data that’s close to

the location

it’s trying to forecast or predict it’s

an unknown territory when that training

data is far away

so known as up by bryant park new york

public library

unknown is down by the one world trade

center

now clearly if i was trying to build a

real drone system

delivery system and ai system for that i

would collect data

all across new york but this is just to

illustrate what happens when we counter

known and unknown

the next thing i need is a model so i’m

going to calibrate a

neural net and i’m going to actually do

this 10 different times

and so what i do is i bring my data into

the computer i push the button

and it goes back and calibrates the no

net and it trains there’s a calcula

bunch of calculations that happen

in the background then it gives me a

formula that can make predictions i can

give any location it will predict the

height of the building

and it’s kind of interesting and really

dirty little secret about

neural nets is every time you push the

button every time you do that regardless

if you have the same data

you’ll get a different formula you’ll

get different predictions so if i do

that 10 different times

i get 10 different predictions and i did

that 10 times so i have 10 neural nets

and i want to show the difference in how

well they do when known territory

versus unknown territory so we’ll start

off by looking

at the new york public library

you can see the red dot there that’s the

top of the new york public library

it’s 190 feet off the ground and if i

take the average of these

10 different predictions it’s not bad

it’s 164 feet

but if i actually plot all 10 of the

different predictions

you’ll see that we have the highest one

at 196 feet the lowest 147 feet

and that range is actually not too bad

given the amount of data i have and i

could arbitrarily shrink that range by

just collecting more data

nearby now

what happens when we move down to a

place that’s unknown territory

one world trade center here the one

world trade center

is 1792 feet above the ground

the average of my 10 neural nets 102

feet

right that’s the little circle with the

blue cross through it

that’s not very good but it gets even

worse

because if you plot all 10 of the

predictions

you’ll see that one of them well you’ve

heard the phrase

off by a mile one of them’s essentially

not only underground but it’s

essentially off by a mile

so first take away from this talk is

that when an ai system encounters the

unknown as far away from its training

data

it’s going to make mistakes mistakes

that could be catastrophic because its

predictions are going to be

catastrophically wrong

now you should say well this is kind of

an easy example in some ways because

it’s easy to know how far away we are

from the trading data

because i only have two numbers that are

coming in the location is just

latitude and longitude right it’s street

address so it’s easy to say that

yes one world trade center is miles away

from where all my training data is

but what happens when your input gets a

lot more complicated that takes us to

our second example

so when you look at images which no nuts

are usually

commonly used in order to do

classifications what you’ve got is

you’ve got

rows and rows and rows of dots and each

one of these dots have got different

numbers

that tell you the color of those dots

and so if you have a high resolution

image you could literally have

millions of inputs this means that the

the model that we calibrate

the formula we come up with is

increased in complexity dramatically and

now he’s increasing complexity

that increasing complexity makes it so

that these models actually are very

fragile

they’re very sensitive to small changes

in data that you might present to it so

let me let me give an example or two

this is our cycle second example

the catch phrase in the academic

literature is adversarial attacks

for these type of examples and what

happens in an adversarial attack

is you imagine that someone’s built a

neural net they’ve

trained it with a bunch of images to do

classification

and then they want to see what small

changes you can do to an image in order

to fool the neural net

so assuming some adversaries trying to

attack this ai system

so the first people are really one of

the first people that really did some of

this work

were a group of researchers in japan and

they put together

a model with a bunch of animals images

and they built a classifier and then

they showed this image of a horse to it

so this image was not in the training

data set

and they asked the classifier to tell

what it was and said it was a horse

then what they went and did is they

actually just changed a single

pixel they said one pixel and they

change it from white to black

and they put this in the classifier and

asked what it was and the classifier

came back and said

that it was a frog so this should be a

little troubling

to you but it shouldn’t be as troubling

as the next example because the next

example

well the next example should be very

troubling to you if you ever think

sometime in the future you might like a

let an ai system take over driving your

car for you so this one focused on

street signs

and classifying street signs like a

bunch of street signs and images of

street signs built a neural net

classifier then it showed it

this image of a stop sign and it was

able to classify it as a stop sign

but then it took the image and just put

a few pieces of tape on it

randomly and asked the neuro net to go

ahead and classify this

and when it came back with the

classification it said it wasn’t a stop

sign it was a go

sign said you go up to 45 miles an hour

now what’s going on in these examples

what’s going on is

their middle the original images are in

some sense

close to the images

that were trained it’s kind of like

being up by bryant park in the drone

example

and the other images we make little

small changes are suddenly far away

and part of the problem with doing image

analysis is really hard to know how far

away two images are because there’s so

many different

parts of the input that you could change

and so

the second takeaway from this talk is

that when

ai systems encounter something new they

don’t know they’ve encountered something

new

ai systems just don’t have a good way of

figuring out when they’re lost

and this is a real problem it’s a

problem we need to think about it’s a

problem that ai’s community

has to find some way of solving so one

obvious solution is to collect more data

but it’s not just

that easy you can’t just go ahead and

say well i’m going to collect more data

because if you collect more data where

you already have data so you collect

more data around bryant park around the

new york public library that doesn’t

help you that much

when you go and encounter something new

like the one world trade center you have

to know when you’ve encountered

something new and then collect the data

what typically happens well what

typically happens is you just do your

best

you build an ai system you collect as

much data as you can and then you run

the ai system

and how do you know that you’ve

encountered something new well there’s a

crash that happens

and i’m not a big fan of this approach

when the ai systems crash

usually that’s bad sometimes it’s

catastrophically bad

and i think we can do a better job i

think we can find a better approach to

that but i have to say that is really

kind of the default approach

that is being used for almost every eye

system out there

so what’s an alternative approach well

alternative approach is to try to mimic

the brain in some way so it’s really

interesting our brain is constantly

making models of the world but what i

find fascinating is it’s actually

doing multiple models so there’s one

model that’s being created by our

brainstem it’s kind of a quick and dirty

model of our world

and there’s another version that’s being

created by our cortex which is more

thoughtful and abstract

and then in the center of our brain

there’s a portion a part of the brain

called the hippocampus and one of its

jobs is to constantly compare these two

models

and see if they line up if they line up

you just kind of carry on

if they don’t line up however you get

this sudden feeling that

that something’s wrong there’s an

anomaly there’s something you need to

focus on

in psychology we call this the orienting

reflex

right so there’s an anomaly and your

brain is a very quick calculation

danger or not if it’s danger you go into

fight-or-flight mode

adrenaline kicks in and

you go into survival mode but if it’s

not dangerous it’s interesting

you still go into a different mode you

go into the explorer mode

because you want to go over and find out

why it is that your two models didn’t

match up you go in there

and collect more data and update your

models

and i think this could offer a really

powerful way

of thinking about how to solve the

problem of helping ai systems

understand when they’re lost

so you could call it maybe the ai

orienting reflex

you run multiple models if the models

agree then you know in your known

territory you carry on

but if the models disagree then you know

you’ve encountered something new

you know that there’s unknown territory

you need to be cautious you need to

actually

slow down right and collect more data

so do ai systems do a good job

when they encounter the unknown no

do a systems know when they’re lost so

far

no is this a problem that we have to

solve absolutely

if we can’t solve this problem there’s a

very real chance the ai

systems will fade just like they did 30

years ago

it’s not just that these ai systems

don’t know when they’re lost

you know that does place a serious limit

on

what we can do with ai systems but

there’s a more fundamental

limit on what ai systems can do

and you’ll notice so far in this talk

i’ve not actually

given a name to ai for most people ai

means artificial intelligence

for me i think that’s misleading i would

prefer to call these systems

augmented intelligence systems and the

reason is that when you think of

artificial intelligence you think of

human cognition values goals

consciousness that’s not what these

systems are what these systems are

at the end of the day really are tools

they’re tools that are powerful they’re

tools that augment our ability

as humans to use our intelligence to

solve problems

and if you think about this difference

it actually will help you understand

the fundamental boundary that ai systems

encounter

and i want to illustrate this by talking

about the problem

or the concept of singularity so the

idea or

maybe it’s better stated the fear of

singularity is eventually

machines are going to get so smart that

they’re going to take over

right from an artificial intelligence

perspective

that’s kind of roughly like saying that

the supercomputer at tesla that does all

the

neural nets and does updates to all the

cars suddenly develops a mind of its own

suddenly

it comes up its own objectives its own

values independent of any human

and who knows what it’ll do maybe it

will go ahead and

and tell the teslas to kick their

drivers out and converge on sturgis

north dakota for a big rally some kind

of big electronic car dance party

from an augmented perspective

intelligence perspective

the singularity risk is different

because

when you have an augmented intelligence

system what it does is it concentrates

more and more control

in the hands of fewer and fewer people

so the risk is something along the lines

of

some teenager in their basement it could

be anywhere it could be russia

china the u.s hacking into that tesla

computer

that supercomputer and taking over and

telling the tesla computer

supercomputer where to send you know to

kick out the

to have the cars kick out their drivers

and go to the big dance party

and that’s the risk i think that in

terms of singularity we should really be

worried about

as a society it’s not the artificial

intelligence risk of the robots taking

over

now is ai here to stay maybe

hopefully if it fades i have no doubt

that it will

most likely come back and the problems

that we’ll solve will be breathtaking

but you shouldn’t be fooled there will

always be a fundamental

limit to ai systems there’ll always be a

boundary and this fundamental

boundary is the ai systems are tools

that means they reflect the values

of the people who created them and who

use them they don’t create their own

values

ultimately the task of determining

what’s valuable what’s moral what’s

ethical

that’s a human task it’s something that

we can never outsource

thank you