We Can All Learn To See Music

i’ve always had

a different perception of music very

different than

classical music theory one of my

earliest recollections of that

difference

i was around age 10 in grade school i

raised my hand and i asked the teacher

why the note on the piano

sounded so yellow the teacher didn’t

really understand the question

she’s a bit annoyed with me today is

very different though

today the sciences of perception are

thriving

and different forms of audio and visual

media

are converging so perhaps today that

question could have drawn curiosity

instead

i found out years later that i have

synesthesia

it’s a condition where the senses are

partially combined

and it’s common to see color

shape and form when listening to music

in my adult life i chose to act on this

quality because i

always suspected that they were

universally visual properties

contained within music visual properties

that everyone can perceive

to some degree after all music

is an anomaly it can draw a highly

emotional response from all of us

and yet there’s nothing tangible in it

like in a painting or in a photograph

so what does music actually mean

and in an increasingly visual world is

just hearing music still enough

what if we could also see music

so i’ve made some animations for today

and i’ve used the same simulations to

produce both the visuals

as well as the sound and through these

animations

i like to convey how the same principles

of design can exist simultaneously

simultaneously in graphics as well as in

music

first size and distance

for many this first diagram might be

obvious i see

size and distance as a basis that’s

incredibly common in both

graphics as well as sound things are

close to us

or larger or if they make forceful

impacts they’ll be louder

and if they’re far away from us or small

or if they make light impacts

they’ll be softer here observe all three

properties

working in unison

[Music]

now let’s get a touch more abstract the

next animation will demonstrate musical

location

instead of depth think of a flat visual

canvas like in a painting

up and down will correspond to the

higher and lower pitch of the note

and left to right will correspond to our

left and right like which ear we hear it

from

[Music]

next the concept of weight there are

strong similarities to how graphic

design

and music approach weight to portray a

heavy weight

large low dim often slow moving elements

are used

and to portray a lighter weight high

bright often more frequent elements are

used

here observe weight working in the

visual sense and at the same time

in the musical sense

[Music]

[Applause]

note progressions or the flow of notes

is a fascinating aspect of music

musicians often describe major and minor

note progressions as

light and dark now let’s compare that to

lighting on a film a scary movie would

be lit with

long dark shadows or nighttime lighting

a kid’s tv show would be completely

different it would use

warm bright light and soft shadows or

daytime lighting

and let’s observe the direct

relationship between lighting and note

progressions

i try to use the piano when i can

because the sound is so recognizable

but for the next two animations i’ve

used direct sound synthesis

to show an even closer relationship

between graphics and sound

let’s have a look at how color

temperature can exhibit the same

properties

as musical timbre

here you might notice how the blue

sounds different than the red

that’s the timbre as the color

temperature increases it goes from blue

to red

and we can also use temperature to

describe musical timbre

a cool timbre would have low distortion

sound waves like a flute

and a warm timbre would have high

distortion sound waves like a violet

[Music]

for the last animation i’d like to

discuss an incredibly foundational

aspect of music repetition patterns of

notes

triplets or notes in pairs of four or

five

graphic design has an equivalent that

everybody knows about

it’s the shape triangles squares

circles and we learn about shapes before

we learn how to walk

i found that musical pitch and

repetition structure

fit seamlessly into the most basic shape

patterns

[Music]

as these animations illustrate there’s

more to music

than abstract patterns and tones and

that the visual

and musical components of our minds

aren’t really so different

so as our perceptions of music continue

to evolve

in an increasingly visual world visual

music theory

will allow us to use our sight to better

understand music

you