Making a car for blind drivers Dennis Hong

many believe driving is an activity

solely reserved for those who can see a

blind person driving a vehicle safely

and independently was thought to be an

impossible task until now hello my name

is Dennis Hong and we’re bringing

freedom and independence to the blind by

building a vehicle for the visually

impaired so before I talk about this car

foot of line let me briefly tell you

about another project I worked on called

the DARPA urban challenge now this was

about building a robotic car that can

drive itself you press Start and nobody

touches anything and can reach his

destination fully autonomously so in

2007 our team won half a million dollars

by placing third place in this

competition so about that time the

National Federation of the blind or NFB

challenged the research community about

who can do up a car that said blind

person drive safely and independently we

decided to give it a try because we

thought hey how hard could it be we just

have a already autonomous vehicle we

just put a blind person in it and we’re

done right we could have been more wrong

what nfe wanted was not a vehicle that

can drive a blind person around but a

vehicle where a blind person can make

active decisions and drive so I had to

throw everything out the window and

start from scratch so to test this crazy

idea we drove a small dune buggy

prototype vehicle to test the Ofili and

in the summer of 2009 we invited dozens

of blind youth from all over the country

and give them a chance to give it for a

spin it was an absolutely amazing

experience but the problem of this car

was it was designed to only be driven in

a very controlled environment and a flat

closed up parking lot even the lane is

defined by red traffic cones so with

this success we decide to take the next

big step to develop a real car that can

be driven on the real roads so how does

it work well it’s a rather complex

system but let me try to explain that

make it simplified so we have three

steps we have

perception computation and non-visual

interfaces now obviously the driver

cannot see so the system needs to

perceive the environment and gather

information for the driver for that we

use a inertial measurement unit so it

measures acceleration angular acts

operation like a human ear inner ear we

fuse that information with a GPS unit to

get an estimate of the location of the

car we also use two cameras to detect

the lanes of the road and we also used

three laser rangefinders the laser scan

the environment to detect obstacles

car approaching from the front the back

and also any obstacles that run into the

roads and the obstacles around the

vehicle so all this vast amount of

information is then fed into the

computer and the computer can do two

things one is first of all process this

information to have an understanding of

the environment these are the lanes of

the road there’s an obstacles and convey

this information to the driver the

system is also smart enough to figure

out the safest way to operate the car so

we can also generate instructions on how

to operate the controls of the vehicle

but the problem is this how do we convey

this information and instructions to a

person who cannot see fast enough and

accurate off so he can drive so for this

we developed many different type of non

visual user interface technologies so

starting from a three dimensional

pinging sound system of vibrating vests

click wheel with voice commands a leg

strip even a shoe that applies pressure

to the foot but today we’re going to

talk about three of these non visual

user interfaces now the first interface

is called a drive grip so these are a

pair of gloves that has vibrating

elements on the knuckle part so it can

convey instructions about how to steer

at the direction and the intensity

another device is called speed strip so

this is a chair as a matter of fact

actually a massage chair we got it out

and we rearrange these arriving elements

and different patterns and we actuate

them to convey information about the

speed and also instructions how to use

the gas and the brake pedal so over here

you can see how the computer understands

the environments and because you cannot

see the vibration we actually put red

LEDs on the drive grip so you can

actually see what’s happening this is

the sensory data and that data is

transferred to the devices through the

computer so these two devices drive grip

and speed strip are very effective but

the problem is these are instructional

cue devices so this is not really

freedom right the computer tells you how

to drive turn left turn right speed up

stop we call this the backseat driver

problem right so we’re moving away from

these instructional cue devices and

now focusing more on the informational

devices a good example for this

informational non-visual user interface

is called air picks so think of it as a

monitor for the blind so it’s a small

tablet has many holes in it and

compressed air comes out so it can

actually draw images so even though

you’re blind put your hand over it you

can see the lanes of the road on

obstacles this actually can also change

the frequency of the air coming out and

possibly the temperatures it’s actually

multi-dimensional user interface so here

you can see in the left camera on the

right camera from the vehicle and how

the computer interprets that and says

that information to the air picks for

this we’re showing a simulator a blind

person driving using the air picks the

simulator was also very useful for

training the mind drivers and also

quickly testing different type of ideas

for a different type of non visual user

interfaces so basically that’s how it

works so just a month ago in January

29th we unveiled this vehicle for the

very first time to the public at the

world-famous Daytona International

Speedway

during the Rolex 24 racing event we also

had some surprises let’s take a look

their first box and now let’s see if

Mark avoid he does he passes it on the

right

there’s boxes out the fourth boxes out

and he’s perfectly making this play to

continue

Mark’s gonna give me a ride back to the

hotel

so since we start this project will be

getting hundreds of letters emails phone

calls from people from all around the

world letters thanking us but sometimes

they also get funny letters like this

one now understand there where there’s

Braille on a drive of ATM machine but

sometimes

but sometimes I also do get you know I

wouldn’t call it hate mail but letters

of really strong concerns dr. Hong are

you insane try to put blight people on

the road you must be out of your mind

but this vehicle is a prototype vehicle

and it’s not gonna be on the road until

it’s proven safe as or safer than

today’s vehicle and I truly believe this

can’t be happen but still with the

society would they accept such a radical

idea how we’re going to handle insurance

how are you going to issue a driving

license there’s many of these kind of

different hurdles besides technology

challenges that we to address before

these firms in reality of course the

main goal of this projects to develop a

car for the blind but potentially more

important than this is the the

tremendous value of the spin-up

technology that can come from this

project the sensors that use can see

through the dark the fog and rain and

together it is new a new type of

interfaces we can use these technologies

that apply to safer cars for sighted

people or for the blind everyday home

appliances in the educational setting in

the office setting just imagine in a

classroom a teacher writes on the

blackboard and a blind student can see

what’s written and reusing these

non-visual interfaces this is priceless

so today the things I’ve showed you

today is just the beginning thank you

very much