What intelligent machines can learn from a school of fish Radhika Nagpal
In my early days as a graduate student,
I went on a snorkeling trip
off the coast of the Bahamas.
I’d actually never swum
in the ocean before,
so it was a bit terrifying.
What I remember the most is,
as I put my head in the water
and I was trying really hard
to breathe through the snorkel,
this huge group
of striped yellow and black fish
came straight at me …
and I just froze.
And then, as if it had
suddenly changed its mind,
came towards me
and then swerved to the right
and went right around me.
It was absolutely mesmerizing.
Maybe many of you
have had this experience.
Of course, there’s the color
and the beauty of it,
but there was also
just the sheer oneness of it,
as if it wasn’t hundreds of fish
but a single entity
with a single collective mind
that was making decisions.
When I look back, I think that experience
really ended up determining
what I’ve worked on for most of my career.
I’m a computer scientist,
and the field that I work in
is artificial intelligence.
And a key theme in AI
is being able to understand intelligence
by creating our own computational systems
that display intelligence
the way we see it in nature.
Now, most popular views of AI, of course,
come from science fiction and the movies,
and I’m personally a big Star Wars fan.
But that tends to be a very human-centric
view of intelligence.
When you think of a fish school,
or when I think of a flock of starlings,
that feels like a really different
kind of intelligence.
For starters, any one fish is just so tiny
compared to the sheer size
of the collective,
so it seems that any one individual
would have a really limited
and myopic view of what’s going on,
and intelligence
isn’t really about the individual
but somehow a property
of the group itself.
Secondly, and the thing
that I still find most remarkable,
is that we know that there are no leaders
supervising this fish school.
Instead, this incredible
collective mind behavior
is emerging purely from the interactions
of one fish and another.
Somehow, there are these interactions
or rules of engagement
between neighboring fish
that make it all work out.
So the question for AI then becomes,
what are those rules of engagement
that lead to this kind of intelligence,
and of course, can we create our own?
And that’s the primary thing
that I work on with my team in my lab.
We work on it through theory,
looking at abstract rule systems
and thinking about
the mathematics behind it.
We also do it through biology,
working closely with experimentalists.
But mostly, we do it through robotics,
where we try to create
our own collective systems
that can do the kinds of things
that we see in nature,
or at least try to.
One of our first robotic quests
along this line
was to create our very own colony
of a thousand robots.
So very simple robots,
but they could be programmed
to exhibit collective intelligence,
and that’s what we were able to do.
So this is what a single robot looks like.
It’s quite small,
about the size of a quarter,
and you can program how it moves,
but it can also wirelessly
communicate with other robots,
and it can measure distances from them.
And so now we can start to program
exactly an interaction,
a rule of engagement between neighbors.
And once we have this system,
we can start to program many
different kinds of rules of engagement
that you would see in nature.
So for example,
spontaneous synchronization,
how audiences are clapping
and suddenly start all clapping together,
the fireflies flashing together.
We can program rules
for pattern formation,
how cells in a tissue
determine what role
they’re going to take on
and set the patterns of our bodies.
We can program rules for migration,
and in this way, we’re really learning
from nature’s rules.
But we can also take it a step further.
We can actually take these rules
that we’ve learned from nature
and combine them and create
entirely new collective behaviors
of our very own.
So for example,
imagine that you had
two different kinds of rules.
So your first rule is a motion rule
where a moving robot can move
around other stationary robots.
And your second rule is a pattern rule
where a robot takes on a color
based on its two nearest neighbors.
So if I start with a blob of robots
in a little pattern seed,
it turns out that these two rules
are sufficient for the group
to be able to self-assemble
a simple line pattern.
And if I have more
complicated pattern rules,
and I design error correction rules,
we can actually create really,
really complicated self assemblies,
and here’s what that looks like.
So here, you’re going to see
a thousand robots
that are working together
to self-assemble the letter K.
The K is on its side.
And the important thing
is that no one is in charge.
So any single robot is only talking
to a small number of robots nearby it,
and it’s using its motion rule
to move around the half-built structure
just looking for a place to fit in
based on its pattern rules.
And even though no robot
is doing anything perfectly,
the rules are such that we can get
the collective to do its goal
robustly together.
And the illusion becomes
almost so perfect, you know –
you just start to not even notice
that they’re individual robots at all,
and it becomes a single entity,
kind of like the school of fish.
So these are robots and rules
in two dimensions,
but we can also think about robots
and rules in three dimensions.
So what if we could create robots
that could build together?
And here, we can take inspiration
from social insects.
So if you think about
mound-building termites
or you think about army ants,
they create incredible,
complex nest structures out of mud
and even out of their own bodies.
And like the system I showed you before,
these insects actually
also have pattern rules
that help them determine what to build,
but the pattern can be made
out of other insects,
or it could be made out of mud.
And we can use that same idea
to create rules for robots.
So here, you’re going to see
some simulated robots.
So the simulated robot has a motion rule,
which is how it traverses
through the structure,
looking for a place to fit in,
and it has pattern rules
where it looks at groups of blocks
to decide whether to place a block.
And with the right motion rules
and the right pattern rules,
we can actually get the robots
to build whatever we want.
And of course, everybody wants
their own tower.
(Laughter)
So once we have these rules,
we can start to create the robot bodies
that go with these rules.
So here, you see a robot
that can climb over blocks,
but it can also lift and move these blocks
and it can start to edit
the very structure that it’s on.
But with these rules,
this is really only one kind of robot body
that you could imagine.
You could imagine
many different kinds of robot bodies.
So if you think about robots
that maybe could move sandbags
and could help build levees,
or we could think of robots
that built out of soft materials
and worked together
to shore up a collapsed building –
so just the same kind of rules
in different kinds of bodies.
Or if, like my group, you are completely
obsessed with army ants,
then maybe one day we can make robots
that can climb over literally anything
including other members of their tribe,
and self-assemble things
out of their own bodies.
Once you understand the rules,
just many different kinds
of robot visions become possible.
And coming back to the snorkeling trip,
we actually understand a great deal
about the rules that fish schools use.
So if we can invent
the bodies to go with that,
then maybe there is a future
where I and my group will get to snorkel
with a fish school of our own creation.
Each of these systems that I showed you
brings us closer to having
the mathematical and the conceptual tools
to create our own versions
of collective power,
and this can enable many different kinds
of future applications,
whether you think about robots
that build flood barriers
or you think about robotic bee colonies
that could pollinate crops
or underwater schools of robots
that monitor coral reefs,
or if we reach for the stars
and we thinking about programming
constellations of satellites.
In each of these systems,
being able to understand
how to design the rules of engagement
and being able to create
good collective behavior
becomes a key to realizing these visions.
So, so far I’ve talked about
rules for insects and for fish
and for robots,
but what about the rules that apply
to our own human collective?
And the last thought
that I’d like to leave you with
is that science is of course itself
an incredible manifestation
of collective intelligence,
but unlike the beautiful
fish schools that I study,
I feel we still have a much longer
evolutionary path to walk.
So in addition to working on improving
the science of robot collectives,
I also work on creating robots
and thinking about rules
that will improve
our own scientific collective.
There’s this saying that I love:
who does science
determines what science gets done.
Imagine a society
where we had rules of engagement
where every child grew up believing
that they could stand here
and be a technologist of the future,
or where every adult
believed that they had the ability
not just to understand but to change
how science and technology
impacts their everyday lives.
What would that society look like?
I believe that we can do that.
I believe that we can choose our rules,
and we engineer not just robots
but we can engineer
our own human collective,
and if we do and when we do,
it will be beautiful.
Thank you.
(Applause)