NVIDIA re-creates Pac-Man using AI, perfect for 40-year anniversary

NVIDIA has worked with Bandai Namco on something truly unique to celebrate the 40th anniversary of the release of a game that changed everything: PAC-MAN.

What did NVIDIA do? NVIDIA researchers trained a new AI model called "NVIDIA GameGAN" that 'watched' 50,000 PAC-MAN episodes to re-produce a fully functional version of the game, all without "an underlying game engine".

How did all of this work? Well, GameGAN, in NVIDIA's own words "the first neural network model that mimics a computer game engine by harnessing generative adversarial networks, or GANs. Made up of two competing neural networks, a generator and a discriminator, GAN-based models learn to create new content that's convincing enough to pass for the original. You may have experienced GANs by using NVIDIA GauGAN, a deep learning model developed by NVIDIA Research that turns rough doodles into photorealistic masterpieces with breathtaking ease".

You will be able to play the game later this year on AI Playground, which you will be able to play here.

The company held a private Q&A with the media, so the NVIDIA researchers answered some questions from the press and we received some more information. NVIDIA said that it used 4 x Tesla V100 GPUs to power the AI-made recreation of PAC-MAN, and had 4 days of training for the AI to work its magic.

NVIDIA's researchers added that the AI is similar to a child, where it gives the AI a simulated environment to work from -- an environment that NVIDIA can control. As it learns, it adapts and gets used to its environment, just like a child will. Children (I'm a father of two) will continue to push boundaries by learning their environments, so an AI is made in a similar fashion.

Where does the company go to in the future with this? GameGAN is an exciting path, as you can use it in something like a factory environment. The same AI tech can be used to teach an AI how a particular factory works, and by feeding it all of the video from the factory and giving it access to controls that robots are handling -- we could have a fully AI-controlled and AI run factory.

I did ask a particular question: "is the AI picking up the SOUND from Pac-Man and re-creating that or just visuals? are you overlaying the Pac-Man sound effects/music from the game, or is the AI actually re-creating EVERYTHING about Pac-Man (at I guess, a code level?)".

NVIDIA's answer was that there is "no audio", but they added that is a "good idea" and they might do that in the future. They confirmed that the AI is recreating just the visuals, and is not generating code -- but also noted that too was a good idea.

So what NVIDIA is doing here is not exactly recreating the game (because it's not taking any code of the game, as I asked that particular question), it is just rather looking at it visually, and recreating it visually. There's no "game" being "recreated".

But, an interesting fact is that during all of this research the AI learned the rules of the original PAC-MAN game, but it learned how to play as PAC-MAN not dying often. So when the AI "recreated" PAC-MAN, it created it in a world where PAC-MAN can't die. So if a human dies in PAC-MAN, they die -- but in the AI-recreated version, it would change the environment and make ghosts go blue so that the player doesn't die.

The researchers noted that this is "kinda like a funny feature", and I agree -- it makes the AI "recreation" of PAC-MAN its own entity.

What could GameGAN do in the future? NVIDIA says it could "accelerate the creative process for game developers, as it could be used to automatically generate layouts for new game levels. GameGan can also be used by AI researchers to more easily develop simulator systems for training autonomous machines, where the AI can learn the rules of an environment before interacting with objects in the real-world".