Augmented Intelligence
[Music]
30 years ago when i was an undergrad
ai suddenly became really popular there
was this wave of enthusiasm that swept
across the campus flipped across the
business world
it seemed like it was only a matter of
time before we were all going to be made
redundant
then aia faltered and it faded
and just went away now it’s back same
enthusiasm same promises
i wonder how well it’s going to do this
time around
how far it’s going to be able to
actually go
all aa systems have got two parts to
them the first thing an ai system has to
do is figure out where it’s at that
means it takes an input
this is data from sensors cameras and it
builds a model of the world
and based on that model of the world it
has to go ahead and decide
what kind of actions it should take and
decisions it has to make
and hopefully these are good but we look
at the
modeling portion it’s really an
important part of the ai system because
the model allows the ai system to make
predictions if the model is
poor then the predictions will be poor
and the ai system is going to end up
making
bad decisions you end up making mistakes
so here’s a couple of questions to think
about with regards to the ass system the
first one is
do a systems how do they do when they
encounter the unknown
and the second one is do ai systems have
any sense
of being lost or when they’re lost so
i’ve got two examples that help explain
and illustrate these answer these
questions the first example
will help us understand what i mean by
an ai system encountering the unknown
the second example will help us see how
easy it is for an ai system to become
lost so my first example i’m assuming
that i’m going to
build a drone delivery system in new
york city so clearly
one of the most important parts of this
is knowing for every location you can be
in new york city
how high you have to fly to make sure
you don’t run into buildings
so i’ve got a map of lower manhattan
this starts top of it is at the bottom
part of central park it goes down to the
southern tip
and i’ve got a couple of points
highlighted here just landmarks
give you some reference first one is new
york public library it’s by bryant park
the second one is the empire state
building a couple of blocks south from
the new york public library
and then a couple of miles down on the
southern tip is the one world trade
center
now this map is actually 3d data it’s a
3d rendering
of all the buildings in new york it was
done by new york city back in 2014
available on the web it’s it’s a pretty
cool set of data
i used this in order to create my
training data because that’s the first
thing i have to have in order to
build an ai system and to build a
predictive model
now i took 50 000 randomly selected
points
locations and then calculated the
heights but i focused all of these
around new york public library so you
can see that kind of concentric
almost target-like set of circles that
shows you where the data is centered
so most of the data is around the new
york public library a little bit around
the empire state building and there’s no
data down by one world trade center
and what i mean by known and unknown for
an ai system
an ai system is in known territory when
it’s got training data that’s close to
the location
it’s trying to forecast or predict it’s
an unknown territory when that training
data is far away
so known as up by bryant park new york
public library
unknown is down by the one world trade
center
now clearly if i was trying to build a
real drone system
delivery system and ai system for that i
would collect data
all across new york but this is just to
illustrate what happens when we counter
known and unknown
the next thing i need is a model so i’m
going to calibrate a
neural net and i’m going to actually do
this 10 different times
and so what i do is i bring my data into
the computer i push the button
and it goes back and calibrates the no
net and it trains there’s a calcula
bunch of calculations that happen
in the background then it gives me a
formula that can make predictions i can
give any location it will predict the
height of the building
and it’s kind of interesting and really
dirty little secret about
neural nets is every time you push the
button every time you do that regardless
if you have the same data
you’ll get a different formula you’ll
get different predictions so if i do
that 10 different times
i get 10 different predictions and i did
that 10 times so i have 10 neural nets
and i want to show the difference in how
well they do when known territory
versus unknown territory so we’ll start
off by looking
at the new york public library
you can see the red dot there that’s the
top of the new york public library
it’s 190 feet off the ground and if i
take the average of these
10 different predictions it’s not bad
it’s 164 feet
but if i actually plot all 10 of the
different predictions
you’ll see that we have the highest one
at 196 feet the lowest 147 feet
and that range is actually not too bad
given the amount of data i have and i
could arbitrarily shrink that range by
just collecting more data
nearby now
what happens when we move down to a
place that’s unknown territory
one world trade center here the one
world trade center
is 1792 feet above the ground
the average of my 10 neural nets 102
feet
right that’s the little circle with the
blue cross through it
that’s not very good but it gets even
worse
because if you plot all 10 of the
predictions
you’ll see that one of them well you’ve
heard the phrase
off by a mile one of them’s essentially
not only underground but it’s
essentially off by a mile
so first take away from this talk is
that when an ai system encounters the
unknown as far away from its training
data
it’s going to make mistakes mistakes
that could be catastrophic because its
predictions are going to be
catastrophically wrong
now you should say well this is kind of
an easy example in some ways because
it’s easy to know how far away we are
from the trading data
because i only have two numbers that are
coming in the location is just
latitude and longitude right it’s street
address so it’s easy to say that
yes one world trade center is miles away
from where all my training data is
but what happens when your input gets a
lot more complicated that takes us to
our second example
so when you look at images which no nuts
are usually
commonly used in order to do
classifications what you’ve got is
you’ve got
rows and rows and rows of dots and each
one of these dots have got different
numbers
that tell you the color of those dots
and so if you have a high resolution
image you could literally have
millions of inputs this means that the
the model that we calibrate
the formula we come up with is
increased in complexity dramatically and
now he’s increasing complexity
that increasing complexity makes it so
that these models actually are very
fragile
they’re very sensitive to small changes
in data that you might present to it so
let me let me give an example or two
this is our cycle second example
the catch phrase in the academic
literature is adversarial attacks
for these type of examples and what
happens in an adversarial attack
is you imagine that someone’s built a
neural net they’ve
trained it with a bunch of images to do
classification
and then they want to see what small
changes you can do to an image in order
to fool the neural net
so assuming some adversaries trying to
attack this ai system
so the first people are really one of
the first people that really did some of
this work
were a group of researchers in japan and
they put together
a model with a bunch of animals images
and they built a classifier and then
they showed this image of a horse to it
so this image was not in the training
data set
and they asked the classifier to tell
what it was and said it was a horse
then what they went and did is they
actually just changed a single
pixel they said one pixel and they
change it from white to black
and they put this in the classifier and
asked what it was and the classifier
came back and said
that it was a frog so this should be a
little troubling
to you but it shouldn’t be as troubling
as the next example because the next
example
well the next example should be very
troubling to you if you ever think
sometime in the future you might like a
let an ai system take over driving your
car for you so this one focused on
street signs
and classifying street signs like a
bunch of street signs and images of
street signs built a neural net
classifier then it showed it
this image of a stop sign and it was
able to classify it as a stop sign
but then it took the image and just put
a few pieces of tape on it
randomly and asked the neuro net to go
ahead and classify this
and when it came back with the
classification it said it wasn’t a stop
sign it was a go
sign said you go up to 45 miles an hour
now what’s going on in these examples
what’s going on is
their middle the original images are in
some sense
close to the images
that were trained it’s kind of like
being up by bryant park in the drone
example
and the other images we make little
small changes are suddenly far away
and part of the problem with doing image
analysis is really hard to know how far
away two images are because there’s so
many different
parts of the input that you could change
and so
the second takeaway from this talk is
that when
ai systems encounter something new they
don’t know they’ve encountered something
new
ai systems just don’t have a good way of
figuring out when they’re lost
and this is a real problem it’s a
problem we need to think about it’s a
problem that ai’s community
has to find some way of solving so one
obvious solution is to collect more data
but it’s not just
that easy you can’t just go ahead and
say well i’m going to collect more data
because if you collect more data where
you already have data so you collect
more data around bryant park around the
new york public library that doesn’t
help you that much
when you go and encounter something new
like the one world trade center you have
to know when you’ve encountered
something new and then collect the data
what typically happens well what
typically happens is you just do your
best
you build an ai system you collect as
much data as you can and then you run
the ai system
and how do you know that you’ve
encountered something new well there’s a
crash that happens
and i’m not a big fan of this approach
when the ai systems crash
usually that’s bad sometimes it’s
catastrophically bad
and i think we can do a better job i
think we can find a better approach to
that but i have to say that is really
kind of the default approach
that is being used for almost every eye
system out there
so what’s an alternative approach well
alternative approach is to try to mimic
the brain in some way so it’s really
interesting our brain is constantly
making models of the world but what i
find fascinating is it’s actually
doing multiple models so there’s one
model that’s being created by our
brainstem it’s kind of a quick and dirty
model of our world
and there’s another version that’s being
created by our cortex which is more
thoughtful and abstract
and then in the center of our brain
there’s a portion a part of the brain
called the hippocampus and one of its
jobs is to constantly compare these two
models
and see if they line up if they line up
you just kind of carry on
if they don’t line up however you get
this sudden feeling that
that something’s wrong there’s an
anomaly there’s something you need to
focus on
in psychology we call this the orienting
reflex
right so there’s an anomaly and your
brain is a very quick calculation
danger or not if it’s danger you go into
fight-or-flight mode
adrenaline kicks in and
you go into survival mode but if it’s
not dangerous it’s interesting
you still go into a different mode you
go into the explorer mode
because you want to go over and find out
why it is that your two models didn’t
match up you go in there
and collect more data and update your
models
and i think this could offer a really
powerful way
of thinking about how to solve the
problem of helping ai systems
understand when they’re lost
so you could call it maybe the ai
orienting reflex
you run multiple models if the models
agree then you know in your known
territory you carry on
but if the models disagree then you know
you’ve encountered something new
you know that there’s unknown territory
you need to be cautious you need to
actually
slow down right and collect more data
so do ai systems do a good job
when they encounter the unknown no
do a systems know when they’re lost so
far
no is this a problem that we have to
solve absolutely
if we can’t solve this problem there’s a
very real chance the ai
systems will fade just like they did 30
years ago
it’s not just that these ai systems
don’t know when they’re lost
you know that does place a serious limit
on
what we can do with ai systems but
there’s a more fundamental
limit on what ai systems can do
and you’ll notice so far in this talk
i’ve not actually
given a name to ai for most people ai
means artificial intelligence
for me i think that’s misleading i would
prefer to call these systems
augmented intelligence systems and the
reason is that when you think of
artificial intelligence you think of
human cognition values goals
consciousness that’s not what these
systems are what these systems are
at the end of the day really are tools
they’re tools that are powerful they’re
tools that augment our ability
as humans to use our intelligence to
solve problems
and if you think about this difference
it actually will help you understand
the fundamental boundary that ai systems
encounter
and i want to illustrate this by talking
about the problem
or the concept of singularity so the
idea or
maybe it’s better stated the fear of
singularity is eventually
machines are going to get so smart that
they’re going to take over
right from an artificial intelligence
perspective
that’s kind of roughly like saying that
the supercomputer at tesla that does all
the
neural nets and does updates to all the
cars suddenly develops a mind of its own
suddenly
it comes up its own objectives its own
values independent of any human
and who knows what it’ll do maybe it
will go ahead and
and tell the teslas to kick their
drivers out and converge on sturgis
north dakota for a big rally some kind
of big electronic car dance party
from an augmented perspective
intelligence perspective
the singularity risk is different
because
when you have an augmented intelligence
system what it does is it concentrates
more and more control
in the hands of fewer and fewer people
so the risk is something along the lines
of
some teenager in their basement it could
be anywhere it could be russia
china the u.s hacking into that tesla
computer
that supercomputer and taking over and
telling the tesla computer
supercomputer where to send you know to
kick out the
to have the cars kick out their drivers
and go to the big dance party
and that’s the risk i think that in
terms of singularity we should really be
worried about
as a society it’s not the artificial
intelligence risk of the robots taking
over
now is ai here to stay maybe
hopefully if it fades i have no doubt
that it will
most likely come back and the problems
that we’ll solve will be breathtaking
but you shouldn’t be fooled there will
always be a fundamental
limit to ai systems there’ll always be a
boundary and this fundamental
boundary is the ai systems are tools
that means they reflect the values
of the people who created them and who
use them they don’t create their own
values
ultimately the task of determining
what’s valuable what’s moral what’s
ethical
that’s a human task it’s something that
we can never outsource
thank you