Toy tiles that talk to each other David Merrill
I want to start by asking you to think
back to when you were a kid playing with
blocks as you figured out how to reach
out and grasp pick them up and move them
around you’re actually learning how to
think and solve problems by
understanding and manipulating spatial
relationships spatial reasoning is
deeply connected to how we understand a
lot of the world around us so as a
computer scientist inspired by this
utility of our interactions with
physical objects along with my adviser
Patty and my collaborator Jeevan kallana
thei I started to wonder what if when we
use a computer instead of having this
one mouse cursor that it was like a
digital fingertip moving around a flat
desktop what if we could reach in with
both hands and grasp information
physically arranging it the way we
wanted this question was so compelling
that we decided to explore the answer by
building SIF tables in a nutshell
acceptable is an interactive computer
the size of a cookie they’re able to be
moved around by hand they can sense each
other they can sense their motion and
they have a screen and a wireless radio
most importantly their physical select
the blocks you can move them just by
reaching out and grasping and the septa
bowls are an example of a new ecosystem
of tools for manipulating digital
information and as these tools become
more physical more aware of their motion
aware of each other and aware of the
nuance of how we move them we can start
to explore some new and fun interaction
styles so I’m going to start with some
simple examples this system was
configured to show video and if I tilt
it in one direction he’ll roll the video
this way if I tilt it the other way it
rolls it backwards and these interactive
portraits are aware of each other so if
I put them next to each other they get
interested if they get surrounded they
notice that - you might get a little
flustered and they can also sense their
motion and tilt so one of the
interesting implications on interaction
we started to realize was that we could
use everyday gestures on data like
pouring a color the way we might pour a
liquid so in this case we’ve got three
septa bowls configured to be paint
buckets and I can use them to pour color
into that central one where they get
mixed if we overshoot we can pour a
little bit back
there are also some neat possibilities
for education like language math and
logic games so you want to give people
the ability to try things quickly and
view the results immediately so here I’m
this is a Fibonacci sequence that I’m
making with a simple equation program
here we have a word game that’s kind of
like a mash-up between Scrabble and
boggle basically in every round you get
a randomly assigned letter on each sip
table and as you try to make words it
checks against the dictionary then after
about 30 seconds it reshuffles and you
have a new set of letters and new
possibilities to try thank you so these
are some kids that came on a field trip
to the Media Lab and I managed to get
them to try it out and shoot a video
they really loved it and one of the
interesting things about this kind of
application is that you don’t have to
give people many instructions all I have
to say is make words and they know
exactly what to do so here’s another few
people trying it out that’s our youngest
beta tester down there on the right
turns out all he wanted to do was to
stack the SIF tables up so to him there
were just blocks now this is an
interactive cartoon application and we
wanted to build a learning tool for for
language learners and this is Felix
actually and he can bring new characters
into the scene just by lifting the SIP
tools off the table that have that
character shown on them here he’s
bringing the Sun out the Sun is rising
now he’s brought a tractor into the
scene
the orange tractor so by shaking the SIP
Sable’s and putting them next to each
other he can make the characters
interact inventing his own narrative
it’s an open-ended story and he gets to
decide how it unfolds so the last
example I have time to show you today is
a music sequencing and live performance
tool that we’ve built recently in which
sift Able’s act as sounds like lead bass
and drums each of these has four
different variations you get to choose
which one you want to use and you can
inject these sounds into a sequence that
you can assemble into the pattern that
you want and you inject it by just
bumping up the sound Cybil against a
sequence if table there are effects that
you can control live like reverb and
filter you attach it to a particular
sound and then tilt to adjust it and
then overall effects like tempo and
volume that apply to the entire sequence
so let’s have a look start by putting a
lead into two sequence if tables arrange
them into a series extend it add a
little more lead now I put a bassline in
now I’ll put some percussion in and now
I’ll attach the filter to the drum so I
can control the effect live I can speed
up the whole sequence by tilting the
tempo to 1 one way or the other and now
I’ll attach the filter to the base for
some more expression I can rearrange the
sequence while it plays I don’t have to
plan it out in advance but I can
improvise changing it making it longer
or shorter as I go and now finally I can
fade the whole sequence out using the
volume shift of all tilted to the left
thank you so
as you can see my passion is for making
new human-computer interfaces that are a
better match to the ways our brains and
bodies work and today I had time to show
you one point in this new design space
and a few of the possibilities that
we’re working to bring out of the
laboratory so the thought I want to
leave you with is that we’re on the cusp
of this new generation of tools for
interacting with digital media that are
going to bring information into our
world on our terms
thank you very much I look forward to
talking with all of you