We Can All Learn To See Music
i’ve always had
a different perception of music very
different than
classical music theory one of my
earliest recollections of that
difference
i was around age 10 in grade school i
raised my hand and i asked the teacher
why the note on the piano
sounded so yellow the teacher didn’t
really understand the question
she’s a bit annoyed with me today is
very different though
today the sciences of perception are
thriving
and different forms of audio and visual
media
are converging so perhaps today that
question could have drawn curiosity
instead
i found out years later that i have
synesthesia
it’s a condition where the senses are
partially combined
and it’s common to see color
shape and form when listening to music
in my adult life i chose to act on this
quality because i
always suspected that they were
universally visual properties
contained within music visual properties
that everyone can perceive
to some degree after all music
is an anomaly it can draw a highly
emotional response from all of us
and yet there’s nothing tangible in it
like in a painting or in a photograph
so what does music actually mean
and in an increasingly visual world is
just hearing music still enough
what if we could also see music
so i’ve made some animations for today
and i’ve used the same simulations to
produce both the visuals
as well as the sound and through these
animations
i like to convey how the same principles
of design can exist simultaneously
simultaneously in graphics as well as in
music
first size and distance
for many this first diagram might be
obvious i see
size and distance as a basis that’s
incredibly common in both
graphics as well as sound things are
close to us
or larger or if they make forceful
impacts they’ll be louder
and if they’re far away from us or small
or if they make light impacts
they’ll be softer here observe all three
properties
working in unison
[Music]
now let’s get a touch more abstract the
next animation will demonstrate musical
location
instead of depth think of a flat visual
canvas like in a painting
up and down will correspond to the
higher and lower pitch of the note
and left to right will correspond to our
left and right like which ear we hear it
from
[Music]
next the concept of weight there are
strong similarities to how graphic
design
and music approach weight to portray a
heavy weight
large low dim often slow moving elements
are used
and to portray a lighter weight high
bright often more frequent elements are
used
here observe weight working in the
visual sense and at the same time
in the musical sense
[Music]
[Applause]
note progressions or the flow of notes
is a fascinating aspect of music
musicians often describe major and minor
note progressions as
light and dark now let’s compare that to
lighting on a film a scary movie would
be lit with
long dark shadows or nighttime lighting
a kid’s tv show would be completely
different it would use
warm bright light and soft shadows or
daytime lighting
and let’s observe the direct
relationship between lighting and note
progressions
so
i try to use the piano when i can
because the sound is so recognizable
but for the next two animations i’ve
used direct sound synthesis
to show an even closer relationship
between graphics and sound
let’s have a look at how color
temperature can exhibit the same
properties
as musical timbre
here you might notice how the blue
sounds different than the red
that’s the timbre as the color
temperature increases it goes from blue
to red
and we can also use temperature to
describe musical timbre
a cool timbre would have low distortion
sound waves like a flute
and a warm timbre would have high
distortion sound waves like a violet
[Music]
for the last animation i’d like to
discuss an incredibly foundational
aspect of music repetition patterns of
notes
triplets or notes in pairs of four or
five
graphic design has an equivalent that
everybody knows about
it’s the shape triangles squares
circles and we learn about shapes before
we learn how to walk
i found that musical pitch and
repetition structure
fit seamlessly into the most basic shape
patterns
[Music]
um
[Music]
oh
as these animations illustrate there’s
more to music
than abstract patterns and tones and
that the visual
and musical components of our minds
aren’t really so different
so as our perceptions of music continue
to evolve
in an increasingly visual world visual
music theory
will allow us to use our sight to better
understand music
you