4 lessons from robots about being human Ken Goldberg

I know this is gonna sound strange but I

think robots can inspire us to be better

humans see I grew up in Bethlehem

Pennsylvania the home of Bethlehem Steel

my father was an engineer and when I was

growing up he would teach me how things

worked we would build projects together

like model rockets and slot cars here’s

the go-kart that we built together

that’s me behind the wheel my sister and

my best friend at the time and one day

he came home when I was about 10 years

old and at the dinner table he announced

that for our next project we were gonna

build a robot a robot now I was thrilled

about this because at school there was a

bully named Kevin and he was picking on

me because I was the only Jewish kid in

class so I couldn’t wait to get started

to work on this so that I could

introduce Kevin to my robot

but that wasn’t the kind of robot my dad

had in mind see he owned a chromium

plating company and they had to move

heavy steel parts between tanks of

chemicals and so he needed an industrial

robot like this that could basically do

the heavy lifting but my dad didn’t get

the kind of robot he wanted either he

and I worked on it for several years but

it was the 1970s and the technology that

was available to amateurs just wasn’t

there yet so dad continued to do this

kind of work by hand and appears later

he was diagnosed with cancer you see the

what the robot we were trying to build

was telling him was not about doing the

heavy lifting it was a warning about his

exposure to the toxic chemicals he

didn’t recognize that at the time and he

contracted leukemia and he died at the

age of 45 I was devastated by this and I

never forgot the robot that he and I

tried to build when I was a college I

decided to study engineering like him

and I went to Carnegie Mellon and I

earned my PhD in robotics

I’ve been studying robots ever since so

what I’d like to tell you about our four

robot projects and what they how they’ve

inspired me to be a better human by 1993

I was a young professor at Kearney at

USC and I was just building up my own

robotics lab and this was the year that

the World Wide Web came out and I

remember my students were the ones who

told me about it and we would we were

just amazed and we started playing with

this and that afternoon we realized that

we could use this new universal

interface to allow anyone in the war

to operate the robot in our lab so

rather than have it fight or do

industrial work we decided to build a

planter put the robot into the center of

it and we called it the telecard and we

put a camera in the gripper of the hand

of the of the robot and we wrote some

special scripts and software so that

anyone in the world could come in and by

clicking on the screen they could move

the robot around and visit the garden

but we also allowed have setup some

other software that lets you participate

and help us water the garden remotely

and if you wanted a few times we’d give

you your own seed to plant and this was

a project an engineering project and we

published some papers on me on the

design the system design of it but we

also thought of it as an art

installation it was invited after the

first year by the ARS electronica museum

in Austria to have it installed in their

Lobby and I’m happy to say it remained

online there 24 hours a day for over

almost nine years that robot was

operated by more people than any other

robot in history now one day I got a

call out of the blue from a student who

asked a very simple but profound

question he said is the robot real now

everyone else had assumed it was and we

knew it was because we were working with

it but I knew what he meant because it

would be possible to take a bunch of

pictures and flowers in a garden and

then basically index them in a computer

system such that it would appear that

there was a real robot when there wasn’t

and the more I thought about it I

couldn’t think of a good answer for how

could he could tell the difference this

was right about this time that I was

offered a position here at Berkeley and

when I got here I looked up Hubert

Dreyfus he was a world-renowned

professor of philosophy

and I talked with him about this and he

said this is one of the oldest and most

central problems in philosophy it goes

back to the sceptics

and up through Descartes it’s the issue

of of epistemology the study of how do

we know that something is true so he and

I started working together and we coined

a new term tell epistemology the study

of knowledge at a distance we invited

leading artists engineers and

philosophers to write essays about this

and the result the results are collected

in this book from MIT press so thanks to

this student who questions what everyone

else had assumed to be true this project

taught me an important lesson about life

which is to always question assumptions

now for the second project I’ll tell you

about grew out of the teller garden as

it was operating my students and I were

very interested in how people were

interacting with each other and what

they were doing with the garden so we

started thinking what if the robot could

leave the garden and go out into some

other interesting environment like for

example what if I could go to a dinner

party at the white house so because we

were interested more in the system

design and the user interface than in

the hardware we decided that rather than

have a robot replaced the human to go to

the party we’d have a human replaced the

robot we called it the telly actor we

got a human someone who’s very outgoing

and gregarious and she was outfitted

with a helmet with various equipment

cameras and microphones and then a

backpack with wireless internet

connection and the idea was that she

could go into a remote and interesting

environment and then over the Internet

people could experience what she was

experiencing so they could see what she

was seeing but then more importantly

they could participate by interacting

with each other and coming up with ideas

about what she should do next

and where she should go and then

conveying those to the telly actor so we

got a chance to take the telly actor to

the Webby Awards in San Francisco and

that year Sam Donaldson was the host

just before the curtain went up I had

about 30 seconds to explain to mr.

Donaldson what we were gonna do I said

the telly actor is a is going to be

joining you on stage and this is a new

experimental project and people are

watching her on their screens and she’s

got there’s cameras involved and there’s

a there’s microphones and she’s got an

earbud in her ear and people over the

network are giving her advice about what

to do next and he said wait a second

that’s what I do

so he loved the concept and when the

tele actor walked on stage she walked

right up to him and she gave him a big

kiss right on the lips

we were totally surprised we had no idea

that would happen and he was great he

just gave her a big hug and returned and

it worked out great but that night as we

were packing up I asked a Tele actor how

did the toilet directors decide that

they would give a kiss to Sam Donaldson

and she said they hadn’t said when she

was just about to walk onstage the

teller directors still were trying to

agree on what to do and so she just

walked onstage and did what felt most

natural

so the success of the telly actor that

night was that it was due to the fact

that she was a wonderful actor she knew

when to trust her instincts and so that

project taught me another lesson about

life which is that when in doubt

improvise

now the third project grew out of the my

experience when my father was in the

hospital he was undergoing a treatment

chemotherapy treatments and there’s a

related treatment called brachytherapy

where tiny radioactive seeds are placed

into the body to treat cancerous tumors

and the way it’s done as you can see

here is that surgeons insert needles

into the body to deliver the seeds and

all this all these needles are inserted

in parallel so it’s very common that

some of the needles penetrate sensitive

organs and as a result the the needles

damaged these organs cause damage which

leads to trauma and side effects so my

students and I wondered what if we could

modify the system so that the needles

could come in at different angles so we

simulated this and we developed some

optimization algorithms and we simulated

this and we were able to show that we

are able to avoid the delicate organs

and yet still achieve the coverage of

the of the tumors with the can’t with

the radiation so now we’re working with

doctors at UCSF and engineers at Johns

Hopkins and we’re building a robot that

has a number of it’s a specialized

design with different joints that can

allow the needles to come in at an

infinite variety of angles and as you

can see here they can avoid delicate

organs - and still reach the targets

they’re aiming for so by questioning

this assumption that all the needles

have to be parallel this project also

taught me an important lesson

when in doubt when your path is blocked

pivot and the last project also has to

do with medical robotics and this is

something that’s grown out of a product

out of a system called The Da Vinci

Surgical robot and this is a

commercially available device that’s

being used in over 2,000 hospitals

around the world and the ideas that

allows the surgeon to operate

comfortably in its own coordinate frame

and it’s but many of the subtasks in

surgery are very routine and tedious

like suturing and currently all these

are performed under the specific and

immediate control of the surgeon so the

surgeon becomes fatigued over time and

we’ve been wondering what if we could

program the robot to perform some of

these subtasks and thereby free the

surgeon to focus on the more complicated

parts of the of the surgery and also cut

down on the time that the surgery would

take if we could get the robot to do

them a little bit faster now it’s hard

to program robot to do delicate things

like this but it turns out my colleague

Peter ABEO who’s here at Berkeley has

developed a new set of techniques for

teaching robots from example so he’s

gotten robots to fly helicopters to

incredibly interesting beautiful

acrobatics by watching human experts fly

them so we got one of these robots we

started working with Peter and his

students and we asked a surgeon to

perform a task and what we do is we

Swift the robot so what we’re doing is

asking the robot the surgeon to perform

the task and we record the motions of

the robot so here’s an example I’ll use

a figure eight tracing out a figure

eight as an example so here’s what it

looks like when the robot this is what

the robots path looks like those three

examples now those are much better than

what a novice like I could do but

they’re still jerky and imprecise so we

record all these examples the data and

then we we go through a sequence of

steps

first we used a technique called dynamic

time warping from speech recognition and

this allows us to temporally align all

the examples and then we apply common

filtering a technique from control

theory that allows us to statistically

analyze all the noise and extract the

desired trajectory that underlies them

now so what we’re doing is we take those

human demonstrations are all noisy and

perfect and we extract from them an

inferred task trajectory and control

sequence for the robot we then execute

that on the robot we observe what

happens then we adjust the controls

using a sequence of techniques called

iterative learning then we do is we

increase the velocity a little bit we

observe the results adjust the controls

again and observe what happens we go

through this several rounds and here’s

the result that’s the inferred trijet

task trajectory and here’s the robot

moving at this compute slew speed of the

human here’s four times the speed of the

human here’s seven times and here’s the

robot operating at ten times the speed

of the human so we’re able to get a

robot to perform a delicate task like I

got a surgical sub task at 10 times the

speed of a human so this project also

because of its involves practicing and

learning doing something over and over

again this project also has a lesson

which is if you wanted to if you want to

do something well there’s no substitute

for practice practice practice so these

are for the lessons that I’ve learned

from robots over the years and 10 and

robotics the field of robotics has

gotten much better over time nowadays

high school students can build robots

like the industrial robot my dad and I

tried to build but it’s very it now and

now I have a daughter named Odessa

she’s eight years old and she likes

robots too maybe it runs in the family I

wish she could meet my dad and I and now

I get to teach her how things work we

get to build projects together and I

wonder what kind of lessons that she’ll

learn from them robots are the most

human of our machines they can’t solve

all of the world’s problems but I think

they have something important to teach

us I invite all of you to think about

the innovations that you’re interested

in the machines that you wish for and

think about what they might be telling

you because I have a hunch that many of

our technological innovations the

devices we dream about can inspire us to

be better humans thank you

you