Toy tiles that talk to each other David Merrill

I want to start by asking you to think

back to when you were a kid playing with

blocks as you figured out how to reach

out and grasp pick them up and move them

around you’re actually learning how to

think and solve problems by

understanding and manipulating spatial

relationships spatial reasoning is

deeply connected to how we understand a

lot of the world around us so as a

computer scientist inspired by this

utility of our interactions with

physical objects along with my adviser

Patty and my collaborator Jeevan kallana

thei I started to wonder what if when we

use a computer instead of having this

one mouse cursor that it was like a

digital fingertip moving around a flat

desktop what if we could reach in with

both hands and grasp information

physically arranging it the way we

wanted this question was so compelling

that we decided to explore the answer by

building SIF tables in a nutshell

acceptable is an interactive computer

the size of a cookie they’re able to be

moved around by hand they can sense each

other they can sense their motion and

they have a screen and a wireless radio

most importantly their physical select

the blocks you can move them just by

reaching out and grasping and the septa

bowls are an example of a new ecosystem

of tools for manipulating digital

information and as these tools become

more physical more aware of their motion

aware of each other and aware of the

nuance of how we move them we can start

to explore some new and fun interaction

styles so I’m going to start with some

simple examples this system was

configured to show video and if I tilt

it in one direction he’ll roll the video

this way if I tilt it the other way it

rolls it backwards and these interactive

portraits are aware of each other so if

I put them next to each other they get

interested if they get surrounded they

notice that - you might get a little

flustered and they can also sense their

motion and tilt so one of the

interesting implications on interaction

we started to realize was that we could

use everyday gestures on data like

pouring a color the way we might pour a

liquid so in this case we’ve got three

septa bowls configured to be paint

buckets and I can use them to pour color

into that central one where they get

mixed if we overshoot we can pour a

little bit back

there are also some neat possibilities

for education like language math and

logic games so you want to give people

the ability to try things quickly and

view the results immediately so here I’m

this is a Fibonacci sequence that I’m

making with a simple equation program

here we have a word game that’s kind of

like a mash-up between Scrabble and

boggle basically in every round you get

a randomly assigned letter on each sip

table and as you try to make words it

checks against the dictionary then after

about 30 seconds it reshuffles and you

have a new set of letters and new

possibilities to try thank you so these

are some kids that came on a field trip

to the Media Lab and I managed to get

them to try it out and shoot a video

they really loved it and one of the

interesting things about this kind of

application is that you don’t have to

give people many instructions all I have

to say is make words and they know

exactly what to do so here’s another few

people trying it out that’s our youngest

beta tester down there on the right

turns out all he wanted to do was to

stack the SIF tables up so to him there

were just blocks now this is an

interactive cartoon application and we

wanted to build a learning tool for for

language learners and this is Felix

actually and he can bring new characters

into the scene just by lifting the SIP

tools off the table that have that

character shown on them here he’s

bringing the Sun out the Sun is rising

now he’s brought a tractor into the

scene

the orange tractor so by shaking the SIP

Sable’s and putting them next to each

other he can make the characters

interact inventing his own narrative

it’s an open-ended story and he gets to

decide how it unfolds so the last

example I have time to show you today is

a music sequencing and live performance

tool that we’ve built recently in which

sift Able’s act as sounds like lead bass

and drums each of these has four

different variations you get to choose

which one you want to use and you can

inject these sounds into a sequence that

you can assemble into the pattern that

you want and you inject it by just

bumping up the sound Cybil against a

sequence if table there are effects that

you can control live like reverb and

filter you attach it to a particular

sound and then tilt to adjust it and

then overall effects like tempo and

volume that apply to the entire sequence

so let’s have a look start by putting a

lead into two sequence if tables arrange

them into a series extend it add a

little more lead now I put a bassline in

now I’ll put some percussion in and now

I’ll attach the filter to the drum so I

can control the effect live I can speed

up the whole sequence by tilting the

tempo to 1 one way or the other and now

I’ll attach the filter to the base for

some more expression I can rearrange the

sequence while it plays I don’t have to

plan it out in advance but I can

improvise changing it making it longer

or shorter as I go and now finally I can

fade the whole sequence out using the

volume shift of all tilted to the left

thank you so

as you can see my passion is for making

new human-computer interfaces that are a

better match to the ways our brains and

bodies work and today I had time to show

you one point in this new design space

and a few of the possibilities that

we’re working to bring out of the

laboratory so the thought I want to

leave you with is that we’re on the cusp

of this new generation of tools for

interacting with digital media that are

going to bring information into our

world on our terms

thank you very much I look forward to

talking with all of you