Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér.
Today we are going to talk about an AI that
not only plays video games really well, but
can also dream up new, unseen scenarios, and
more.
This is an interesting new framework that
contains a vision model that compresses what
it has seen in the game into an internal code.
As you see here, these latent variables are
responsible to capture different level designs,
and this variable simulates time and shows
how the fireballs move towards us over time.
This is a highly compressed internal representation
that captures the most important aspects of
the game.
We also have a memory unit that not only stores
previous experiences, but similarly to how
an earlier work predicted the next pen strokes
of a drawing, this can also dream up new gameplay.
Finally, it is also endowed with a controller
unit that is responsible for making decisions
as to how to play the game.
Here, you see the algorithm in action: on
the left, there is the actual gameplay, and
on the right you can see its compressed internal
representation.
This is how the AI thinks about the game.
The point is that it is lossy, therefore some
information is lost, but the essence of the
game is retained.
So, this sounds great, the novelty is clear,
but how well does it play the game?
Well, in this racing game, on a selection
of a 100 random tracks, its average score
is almost three times that of DeepMind's groundbreaking
Deep Q-Learning algorithm.
This was the AI that took the world by storm
when DeepMind demonstrated how it learned
to play Atari Breakout and many other games
on a superhuman level.
This is almost three times better than that
on the racetrack game, though it is to be
noted that DeepMind has also made great strides
since their original DQN work.
And now comes the even more exciting part!
Because it can create an internal dream representation
of the game, and this representation really
captures the essence of the game, then it
means that it is also be able to play and
train within these dreams.
Essentially, it makes up dream scenarios and
learns how to deal with them without playing
the actual game.
It is a bit like how a we prepare for a first
date, imagining what to say, and how to say
it, or, imagining how we would incapacitate
an attacker with our karate chops if someone
were to attack us.
And the cool thing is that with this AI, this
dream training actually works, which means
that the newly learned dream strategies translate
really well to the real game.
We really have only scratched the surface,
so make sure to read the paper in the description.
This is a really new and fresh idea, and I
think it will give birth to a number of followup
papers.
Cannot wait to report on these back to you,
so stay tuned and make sure to subscribe and
hit the bell icon to never miss an episode.
Thanks for watching and for your generous
support, and I'll see you next time!
