Making music with your mind

[Music]

think about the importance of music in

your life

do you play an instrument do you

sing in the shower do you

love to listen to your favorite music

what if we could take that joy of music

whatever it is for an individual and use

it to restore

function in those who are paralyzed

in college i was a musician

who really loved science or maybe i was

a scientist who

really loved music as more than just a

hobby

i couldn’t reconcile these two things

because

for the academic world they seem like

chocolate and peanut butter they just

they don’t go together so

after first declaring a major in music

and music composition

i later switched to molecular biology

and i continued to pursue music on the

side it just it seemed like the

practical thing to do

then later in graduate school i was

doing

a phd in neurobiology

on the genetics of brain development

i continued to pursue music on the side

i

played in bands and i studied music

i got a certificate in jazz at a

conservatory

and for me music was then

and still is today always will be

absolutely

essential for me as a way to express my

emotion to the world

in a way that makes sense to me it’s

healing my heart and it is restorative

to my mind

but again to the academic world it

seemed

to paint me as kind of a jekyll and hyde

character or maybe

black swan well ten years ago

uh i was doing a neurology fellowship

in surgical epilepsy so in the

laboratory i was studying

the musical processing in the brain

the electrical traces of complex sound

analysis

and then in my art and music studio

i was creating an art project which i

called the encephalophone so

this was a brain computer interface that

allowed people who

couldn’t move to play music again so

it allowed people to play music without

movement from brain

signal directly then as a neurologist

i’m in the hospital and i’m seeing

stroke patients and i began noticing in

some of these

stroke patients a loss of not just their

ability to move

or their ability to speak but

a largely unrecognized

loss of quality of life a inability to

play music and for

professional musicians and in even

enthusiastic amateurs this is

pretty devastating it’s a major loss of

quality of life

and a light bulb went off my head and i

said

oh i can take this art project i have

and i could use it to actually restore

that quality of life for these patients

for these people

and so these parallel paths

were finally starting to merge and come

together

so how does this work how does the

encephalophone work so

what we do is we measure the brain

activity

in a part of the brain called the motor

cortex so this is the part of the brain

that tells your arms and legs to move

and we look at that activity

when you move your arm that activity

goes up but if you

think about moving your arm what’s neat

is that actually

has the same activity so we can measure

that when you’re thinking about moving

but you don’t actually have to move

let’s try something here

to give you an example of that i want

you to close your eyes

and just relax and imagine you’re

floating in the water

and you’re looking up at the clouds in

the sky

so just relax and float

and watch the clouds in the sky go by

you’re now playing high notes on the

encephalophone

so now i want you to imagine moving your

right hand

just imagine moving it don’t actually

move it

think about taking a squeeze ball and

just squeeze and unsqueeze grip and

ungrip

now you’re playing the low notes on the

encephalophone

so open your eyes so what we’re doing is

we’re measuring in that motor cortex

that part that tells your arms and legs

to move

when you’re relaxed thinking of the

clouds in the sky

you’re getting high notes when you’re

thinking of gripping and ungripping that

squeeze ball

you’re getting the low notes so it’s a

bit

challenging and awkward at first just

like any

new musical instrument but the joy it

can bring to people who

are empowered to play music again is

just unmistakably powerful

so just to give you an example i had a

patient in the hospital

maria she was paralyzed from the neck

down couldn’t even speak

but before all this she was a musician

she played in bands

she wrote music she sang

and then she got a brain tumor and

multiple surgeries

and she’s completely devastated can’t

move can’t speak

and can’t play music so in the clinical

trials in the hospital

she began using the encephalophone and

when she got to that moment when you

could see that she was realizing

that she was actually playing music and

had control

she began to smile she began to giggle

and she began to cry in joy at being

at this moment where she realized i’m

playing music again for the first time

in eight years

so these clinical trials that i was

doing

they came out of a desire to show that

the encephalophone wasn’t just a

a novelty art project but it actually

worked

so in order to show that it worked i

needed to do

some experiments in the laboratory

so what we first did was we took

normal healthy individuals people

without any motor disability some of

them

musicians some of them non-musicians

and we had them use the encephalophone

to try to match a target note so we gave

them a note

and they’re playing the note doing what

you were just doing a few minutes ago

relaxing thinking the clouds in the sky

to bring the note up higher

or thinking about squeezing that squeeze

ball to bring the notes down lower

and they needed to try to match the note

three times in a row

well with two different methods we had

all these people 15 subjects all of them

not only had

accuracy and had real control but they

actually had

much better accuracy than random so this

was really exciting it showed that it

wasn’t just

random notes coming out of their brain

but it was actually

real control it really worked so

now that i’d shown that it worked for

normal individuals i wanted to show

this could work for people like maria

who had

motor disability people with stroke

people with brain hemorrhages als

multiple sclerosis spinal cord injury

amputees from military veterans

all types of people who have real motor

disabilities so

we did the same type of experiment where

we had them match the target note

but these were people who couldn’t even

move so not only

did they match the target note with some

accuracy

they actually improved over time so over

these six trials

they were able to they were actually

learning they were learning how to do

this better

so that was really exciting and

satisfying

out of all those 12 patients one of

those patients was jonathan

so jonathan is a brilliant computer

programmer and

musician who has multiple sclerosis

so his form of multiple sclerosis

affected his brain stem

not his brain so he’s completely intact

in his brain

but ms has basically cut the connections

between the motor cortex that i was just

talking about the part that tells your

arms and legs to move

it’s cut the connection between his

brain and his body

so the way he describes it in his own

words are

he says i imagine the disease like a

slippage

slowly losing contact i have with the

physical world

so the scientist in me motivated me to

do these experiments i was very

satisfied because i got to show

that this not only worked for normal

people

but it worked for people like jonathan

and maria who

actually can’t move that’s really

exciting but

the musician in me needed to bring this

to performances

when they start performing these

individuals

are no longer patients so they become

performing musicians and they’re able to

connect with the audience

and actually be able to create music

again and share that with other people

so i put on a series of concerts and one

of the concerts in 2019

showed two quadriplegic musicians and

one of them was jonathan

playing with a live jazz ensemble

i’d like to show you a clip of that

[Music]

[Applause]

so this performance brought myself and

the band

and the entire audience to tears

but the words that jonathan used to

describe the experience in his own words

really touched me just as much so he

said

the slippage that the disease causes me

to lose connection with the physical

world it’s like a curtain coming down

over the stage

and the performance with the

encephalophone that night

opened up that curtain just a little bit

and allowed me to peek through

and make a real connection with the

audience it was

that that made it a really magical

experience for me

so we’ve been able to show that

we can re-empower people who have motor

disability to play music again

and perform as musicians once more

but what if we could take the

encephalophone and not just

allow people to play music but we could

actually get people who are

paralyzed to move again what if we could

take those parts of their brain that’s

damaged and dormant and repair it so

that they could actually start moving

again we’re going to do

experiments to try to show just that so

we’re going to try to show if the

encephalophone can make an

actual therapeutic or structural

difference

we’ll look at motor skills improvement

see if they can move better

cognitive improvement see if they can

think better

and we’ll look at rewiring through mri

sequences

so this would be looking at the wiring

seeing if those dormant parts by being

stimulated

can then be rewired and repaired so that

someone who

maybe couldn’t pick up a cup of coffee

or even feed themselves could do that

again

so we’ve made these connections we’ve

made connections to these

patients and then performers and the

audience

and this talk and these videos i may be

able to connect

to more people but i’d like to share

this device with

a much wider audience by making a device

that would be widely available

this way someone could anyone anywhere

who has motor disability

could take the device home and they

could be empowered

to create music again anywhere

so why is this important well it’s

probably pretty clear to you

by now that music is very important to

me but why is music important in the

world well

music is not just a cultural cornerstone

of every

society in human history

it’s actually a behavior that’s wired

into the structure of every human

being’s brain

music is not just for entertainment it’s

critical in the development of the brain

and for learning

emotional communication so by empowering

people

to play music who’d never been able to

move before it’s not only empowering but

it’s key to their

participation in a truly full life

thank you