Making music with your mind
[Music]
think about the importance of music in
your life
do you play an instrument do you
sing in the shower do you
love to listen to your favorite music
what if we could take that joy of music
whatever it is for an individual and use
it to restore
function in those who are paralyzed
in college i was a musician
who really loved science or maybe i was
a scientist who
really loved music as more than just a
hobby
i couldn’t reconcile these two things
because
for the academic world they seem like
chocolate and peanut butter they just
they don’t go together so
after first declaring a major in music
and music composition
i later switched to molecular biology
and i continued to pursue music on the
side it just it seemed like the
practical thing to do
then later in graduate school i was
doing
a phd in neurobiology
on the genetics of brain development
i continued to pursue music on the side
i
played in bands and i studied music
i got a certificate in jazz at a
conservatory
and for me music was then
and still is today always will be
absolutely
essential for me as a way to express my
emotion to the world
in a way that makes sense to me it’s
healing my heart and it is restorative
to my mind
but again to the academic world it
seemed
to paint me as kind of a jekyll and hyde
character or maybe
black swan well ten years ago
uh i was doing a neurology fellowship
in surgical epilepsy so in the
laboratory i was studying
the musical processing in the brain
the electrical traces of complex sound
analysis
and then in my art and music studio
i was creating an art project which i
called the encephalophone so
this was a brain computer interface that
allowed people who
couldn’t move to play music again so
it allowed people to play music without
movement from brain
signal directly then as a neurologist
i’m in the hospital and i’m seeing
stroke patients and i began noticing in
some of these
stroke patients a loss of not just their
ability to move
or their ability to speak but
a largely unrecognized
loss of quality of life a inability to
play music and for
professional musicians and in even
enthusiastic amateurs this is
pretty devastating it’s a major loss of
quality of life
and a light bulb went off my head and i
said
oh i can take this art project i have
and i could use it to actually restore
that quality of life for these patients
for these people
and so these parallel paths
were finally starting to merge and come
together
so how does this work how does the
encephalophone work so
what we do is we measure the brain
activity
in a part of the brain called the motor
cortex so this is the part of the brain
that tells your arms and legs to move
and we look at that activity
when you move your arm that activity
goes up but if you
think about moving your arm what’s neat
is that actually
has the same activity so we can measure
that when you’re thinking about moving
but you don’t actually have to move
let’s try something here
to give you an example of that i want
you to close your eyes
and just relax and imagine you’re
floating in the water
and you’re looking up at the clouds in
the sky
so just relax and float
and watch the clouds in the sky go by
you’re now playing high notes on the
encephalophone
so now i want you to imagine moving your
right hand
just imagine moving it don’t actually
move it
think about taking a squeeze ball and
just squeeze and unsqueeze grip and
ungrip
now you’re playing the low notes on the
encephalophone
so open your eyes so what we’re doing is
we’re measuring in that motor cortex
that part that tells your arms and legs
to move
when you’re relaxed thinking of the
clouds in the sky
you’re getting high notes when you’re
thinking of gripping and ungripping that
squeeze ball
you’re getting the low notes so it’s a
bit
challenging and awkward at first just
like any
new musical instrument but the joy it
can bring to people who
are empowered to play music again is
just unmistakably powerful
so just to give you an example i had a
patient in the hospital
maria she was paralyzed from the neck
down couldn’t even speak
but before all this she was a musician
she played in bands
she wrote music she sang
and then she got a brain tumor and
multiple surgeries
and she’s completely devastated can’t
move can’t speak
and can’t play music so in the clinical
trials in the hospital
she began using the encephalophone and
when she got to that moment when you
could see that she was realizing
that she was actually playing music and
had control
she began to smile she began to giggle
and she began to cry in joy at being
at this moment where she realized i’m
playing music again for the first time
in eight years
so these clinical trials that i was
doing
they came out of a desire to show that
the encephalophone wasn’t just a
a novelty art project but it actually
worked
so in order to show that it worked i
needed to do
some experiments in the laboratory
so what we first did was we took
normal healthy individuals people
without any motor disability some of
them
musicians some of them non-musicians
and we had them use the encephalophone
to try to match a target note so we gave
them a note
and they’re playing the note doing what
you were just doing a few minutes ago
relaxing thinking the clouds in the sky
to bring the note up higher
or thinking about squeezing that squeeze
ball to bring the notes down lower
and they needed to try to match the note
three times in a row
well with two different methods we had
all these people 15 subjects all of them
not only had
accuracy and had real control but they
actually had
much better accuracy than random so this
was really exciting it showed that it
wasn’t just
random notes coming out of their brain
but it was actually
real control it really worked so
now that i’d shown that it worked for
normal individuals i wanted to show
this could work for people like maria
who had
motor disability people with stroke
people with brain hemorrhages als
multiple sclerosis spinal cord injury
amputees from military veterans
all types of people who have real motor
disabilities so
we did the same type of experiment where
we had them match the target note
but these were people who couldn’t even
move so not only
did they match the target note with some
accuracy
they actually improved over time so over
these six trials
they were able to they were actually
learning they were learning how to do
this better
so that was really exciting and
satisfying
out of all those 12 patients one of
those patients was jonathan
so jonathan is a brilliant computer
programmer and
musician who has multiple sclerosis
so his form of multiple sclerosis
affected his brain stem
not his brain so he’s completely intact
in his brain
but ms has basically cut the connections
between the motor cortex that i was just
talking about the part that tells your
arms and legs to move
it’s cut the connection between his
brain and his body
so the way he describes it in his own
words are
he says i imagine the disease like a
slippage
slowly losing contact i have with the
physical world
so the scientist in me motivated me to
do these experiments i was very
satisfied because i got to show
that this not only worked for normal
people
but it worked for people like jonathan
and maria who
actually can’t move that’s really
exciting but
the musician in me needed to bring this
to performances
when they start performing these
individuals
are no longer patients so they become
performing musicians and they’re able to
connect with the audience
and actually be able to create music
again and share that with other people
so i put on a series of concerts and one
of the concerts in 2019
showed two quadriplegic musicians and
one of them was jonathan
playing with a live jazz ensemble
i’d like to show you a clip of that
[Music]
[Applause]
so this performance brought myself and
the band
and the entire audience to tears
but the words that jonathan used to
describe the experience in his own words
really touched me just as much so he
said
the slippage that the disease causes me
to lose connection with the physical
world it’s like a curtain coming down
over the stage
and the performance with the
encephalophone that night
opened up that curtain just a little bit
and allowed me to peek through
and make a real connection with the
audience it was
that that made it a really magical
experience for me
so we’ve been able to show that
we can re-empower people who have motor
disability to play music again
and perform as musicians once more
but what if we could take the
encephalophone and not just
allow people to play music but we could
actually get people who are
paralyzed to move again what if we could
take those parts of their brain that’s
damaged and dormant and repair it so
that they could actually start moving
again we’re going to do
experiments to try to show just that so
we’re going to try to show if the
encephalophone can make an
actual therapeutic or structural
difference
we’ll look at motor skills improvement
see if they can move better
cognitive improvement see if they can
think better
and we’ll look at rewiring through mri
sequences
so this would be looking at the wiring
seeing if those dormant parts by being
stimulated
can then be rewired and repaired so that
someone who
maybe couldn’t pick up a cup of coffee
or even feed themselves could do that
again
so we’ve made these connections we’ve
made connections to these
patients and then performers and the
audience
and this talk and these videos i may be
able to connect
to more people but i’d like to share
this device with
a much wider audience by making a device
that would be widely available
this way someone could anyone anywhere
who has motor disability
could take the device home and they
could be empowered
to create music again anywhere
so why is this important well it’s
probably pretty clear to you
by now that music is very important to
me but why is music important in the
world well
music is not just a cultural cornerstone
of every
society in human history
it’s actually a behavior that’s wired
into the structure of every human
being’s brain
music is not just for entertainment it’s
critical in the development of the brain
and for learning
emotional communication so by empowering
people
to play music who’d never been able to
move before it’s not only empowering but
it’s key to their
participation in a truly full life
thank you