We need a moral operating system Damon Horowitz
power that is the word that comes to
mind we’re the new technologists we have
a lot of data so we have a lot of power
how much power do we have seen from a
movie Apocalypse Now great movie we got
to get our hero captain Willard to the
mouth of the number so we can go pursue
Colonel Kurtz the way we’re going to do
this just fly him in and drop them off
so the scene the sky is filled with this
fleet of helicopters carrying him in and
there’s this loud thrilling music in the
background it’s wild music
don’t knock on door knock on door aw
that’s a lot of power that’s the kind of
power I feel in this room that’s the
kind of power we have because all of the
data that we have let’s take an example
what can we do with just one person’s
data what can we do with that guy’s data
I can look at your financial records I
can tell if you pay your bills on time I
know if you’re good to give a loan to
can look at your medical records I can
see if the pump is still pumping see if
you’re good to offer insurance to I can
look at your clicking patterns when you
come to my website I actually know what
you’re gonna do already because I’ve
seen you visit millions of websites
before and I’m sorry to tell you you’re
like a poker player you ever tell I can
tell what’s data analysis what you’re
going to do before you even do it I know
what you like I know who you are and
that’s even before I look at your mail
or your phone those are the kind of
things we can do with the data that we
have but I’m not actually here to talk
about what we can do I’m here to talk
about what we should do what’s the right
thing to do now I see some puzzled looks
like why are you asking us what’s the
right thing to do we’re just building
this stuff somebody else is using it
fair enough but it brings me back think
about world war two some of our great
technologists and some of our great
physicists studying
nuclear fission fusion just nuclear
stuff we gathered together these
physicists and Los Alamos see what
they’ll build we want the people
building the technology thinking about
what we should be doing with the
technology so what’s we doing with that
guys data should be collecting it
gathering it so we can make his online
experience better so we can make money
so we can protect ourselves if you put
up to no good or should we respect his
privacy protect his dignity and leave
him alone which one is it how should we
figure it out
I know crowdsource that’s crowdsource
this so to get people warmed up let’s
start with it’s not an easy question
something I’m sure everybody here has an
opinion about iPhone versus Android
let’s do the show hands iPhone uh-huh
Android you’d think with a bunch of
smart people we wouldn’t be such suckers
just for the pretty phones okay next
question a little bit harder should we
be collecting all of that guy’s data to
make his experiences better and to
protect ourselves in case it up to no
good or should we leave him alone
collect his data leave him alone
you’re safe it’s high okay last question
harder question when trying to evaluate
what we should do in this case should we
use a Content deontological amoral
framework or should we use a militant a
million consequentialist one can’t milk
not not as many votes yeah that’s a
terrifying result it’s terrifying
because we have stronger opinions about
our handheld devices then about the
moral framework we should use to guide
our decisions how do we know what to do
with all the power we have if we don’t
have a moral framework we know more
about mobile operating systems but what
we really need is a moral operating
system
what’s a moral operating system all
right we all know right and wrong right
you feel good when you do something
right if you’re bad when you do
something wrong
our parents teach us that you know
praise with a good it’s gold with a bad
but how do we figure out what’s right
and wrong
I’m from day to day we have the
techniques that we use right
maybe we just follow our gut maybe we
take a vote we crowdsource maybe we punt
ask the legal department see what they
say
in other words it’s kind of random kind
of ad hoc how we figure out what we
should do and maybe if we want to be
unsure or footing what we really want is
a moral framework that will help guide
us there that will tell us what kinds of
things are right and wrong in the first
place and how would we know in a given
situation what to do so let’s get a
moral framework we’re numbers people
living by numbers how can we use numbers
as the basis for a moral framework I
know a guy who did exactly that
brilliant guy
he’s been dead 2,500 years
Plato that’s right remember him old
philosopher you were sleeping during
that class umm Plato he had a lot of the
same concerns that we did he was worried
about right and wrong he wanted to know
what is just but he was worried that all
that we seem to be doing is trading
opinions about this he says something’s
just she says something else is just
it’s kind of convincing when he talks
when she talks to I’m just going back
and forth I’m not getting anywhere I
don’t want opinions I want knowledge I
want to know the truth about justice
like we have truths in math
in math we know the objective facts take
a number any no to favorite number I
love that number um there are truths
about two if you got two of something
you add two more you get four that’s
true no matter what thing you’re talking
about it’s an objective truth about the
form of two the abstract form we have
two of anything two eyes two ears two
noses just two protrusions those will
partake of the form of two they all
participate in the truths the two has
they’ll have tunas in them and therefore
it’s not a matter of opinion
what if Plato thought ethics was like
math what if there were a pure form of
justice what if there are truths about
justice and you could just look around
in this world and see which things
participated per took of that form of
justice then you would know what was
really just and what wasn’t it wouldn’t
be a matter of just opinion or just
appearances that’s a stunning vision um
think about that how grand how ambitious
that’s that’s like exam be sure says we
are he wants to solve ethics what
subjective truth if you if you think
that way you have a plate inist
moral framework if you don’t think that
way well you have a lot of company in
the history of Western philosophy
because the tidy idea you know people
criticized it Aristotle was in
particular he was not amused
he thought it was Impractical Aristotle
said we should seek only so much
precision in each subject as that
subject allows Aristotle thought ethics
wasn’t a lot like math he thought ethics
was a matter of making decisions in the
here and now using our best judgment to
find the right path if you think that
Plato is not your guy but don’t give up
maybe there’s another way that we can
use numbers as the basis of our moral
framework how about this what if in any
situation you could just calculate look
at the choices measure out which one’s
better and know what to do that sound
familiar
yep that’s a utilitarian moral framework
John Stuart Mill is a great advocate of
this nice guy besides
and only been dead 200 years um so basis
of utilitarianism I’m sure you’re
familiar or at least the three people
who voted for meal before familiar with
this but here’s what works what if
morals what if what makes something
moral it’s just a matter of if it
maximizes pleasure and minimizes pain it
that’s something intrinsic to the act
it’s not like its relation to some
abstract form it’s just a matter of the
consequences you just look at the
consequences and see if overall it’s for
the good or further worse that would be
simple then we know what to do let’s
take an example suppose I go up and I
say I’m going to take your phone not
just because it rang earlier but I’m
going to take it because I made a little
calculation I thought that guy looks
suspicious and what if he’s been sending
little messages to bin Laden’s hideout
or whoever took over after bin Laden and
he’s actually like a terrorist like a
sleeper so I’m gonna find that out when
I find that out I’m gonna prevent a huge
amount of damage that he could cause
that has a very high utility to prevent
that damage and compare it to the little
pain that it’s going to cause it’s gonna
be embarrassing when I’m looking on his
phone and seeing as a farm bill problem
and that whole bit that’s over
overwhelmed by the value of looking at
the problem if you feel that way that’s
a utilitarian choice but maybe you don’t
feel that way either maybe you think
it’s his phone it’s wrong to take his
phone because he’s a person and he has
rights and he has dignity and we can’t
just interfere with that he has autonomy
it doesn’t matter what the calculation
are there are things that are
intrinsically wrong like Wyman is wrong
like torturing innocent children is
wrong
Kant was very good on this point and he
said it a little better than I’ll say he
said we should use our reason to figure
out the rules by which we should guide
our conduct and then it is our duty to
follow those rules it’s not a matter of
calculation - let’s stop we’re right in
the thick of it this philosophical
thicket and this goes on for thousands
of years because these are hard
questions and I’ve only got 15 minutes
so let’s cut to the chase
how should we be making our decisions is
it
Plato’s and Aristotle’s it can’t isn’t
mill what should we be doing what’s the
answer what’s the formula that we can
use in any situation to determine what
we should do whether we should use that
guy’s data or not what’s the formula
there’s not a formula there’s not a
simple answer
ethics is hard ethics requires thinking
and that’s uncomfortable I know I spent
a lot of my career in artificial
intelligence trying to build machines
that could do some of this thinking for
us they could give us answers but but
they can’t you can’t just take human
thinking and put it into a machine we’re
the ones who have to do it happily we’re
not machines and we can do it and then
we can we think we must Hana Aaron said
the sad truth is that most evil done in
this world is not done by people who
choose to be evil it arises from not
thinking that’s what we call the
banality of evil and the response to
that is that we demand the exercise of
thinking from every sane person so let’s
do that let’s think in fact let’s start
right now every person in this room do
this think of the last time you had a
decision to make where you were worried
to do the right thing were you wondered
about what should I be doing bring that
to mind and now reflect on that and say
how did I come up with that decision
what should I do did I follow my gut did
I have somebody vote on it did I punt
illegal or now we have a few more
choices did I evaluate what would be the
highest pleasure like Millwood or like
contem I use reason to figure out what
was intrinsically right think about it
really bring it to mind this is
important it is so important we are
going to spend 30 seconds of valuable
TED talk time doing nothing but thinking
about this are you ready go
stop good work what you just did that’s
the first step towards taking
responsibility for what we should do
with all of our power
now the next step try this
go find a friend and explain to them how
you made that decision not right now
wait till I finish talking do it over
lunch um and don’t just find another
technologist friend find somebody
different than you find an artist or a
writer or heaven forbid find a
philosopher and talk to them in fact
find find somebody from the humanities
why because they think about problems
differently than we do as technologists
I mean just a few days ago right across
the street from here there’s hundreds of
people gathered together
it was technologists and humanists at
that big bibliothèque conference and
they gathered together because the
technologists wanted to learn what it
would be like to think from a humanities
perspective you have someone from google
talking to somebody who does comparative
literature you’re thinking about the
relevance of 17th century french theater
how does that bear upon venture capital
right well that’s interesting that’s a
different way of thinking and when you
think in that way you become more
sensitive to the human considerations
which are crucial to making ethical
decisions so imagine that right now you
went and you found your musician friend
and you’re telling them what we’re
talking about you know about our whole
data revolution all this maybe even like
hum a few bars of our theme music dum
dum dum dum dum well your musician
friend will stop you and say you know
the theme music for your data revolution
that’s an opera that’s vogner’s based on
Norse legend it’s God’s and mythical
creatures fighting over magical jewelry
that’s interesting now it’s also a
beautiful opera and we’re moved by that
opera removed because it’s about the
battle between good and evil about right
and wrong and we care about right and
wrong we care what happens in that opera
we care what happens in Apocalypse Now
and we certainly care what happens with
our technology
we have so much power today it is up to
us to figure out what to do and that’s
the good news
we’re the ones writing this opera this
is our a movie we figure out what will
happen with this technology we determine
how this will all end thank you