There are people who have played so much Grand theft auto v who know the map by heart. But it is very different for a human to learn a map than for a neural network to memorize it to create its own version just by observing games.
Behind this recreation of part of the map is not the game engine or anything, but there is an artificial intelligence creating the spaces and images of the game world using a model called GameGAN from NVIDIA. This, in turn, is based on what is called the Generative Adversary Network (GAN), which in the past has been used to create human-like faces.
This version of GTA V -call appropriately GANTheft Auto- is built by collecting data by the neural network, which observes what happens in the games and absorbs said knowledge and then create your own interpretation. This version is founded not only on the visual, but also on the buttons that are pressed, the physical interaction of objects within the game, the environment represented on the map and so on.
The project is the work of two researchers, Harrison Kinsley and Daniel Kukeila, who explain that no line of code was written to create this, nor was any rule of how this representation should work; the neural network draws the images and responds to the commands according to the previous training.
That training was done on an NVIDIA workstation called DGX A100, designed precisely for machine learning systems (also called machine learning). The GAN observed the behavior of 12 bots playing Grand Theft Auto V, until you get the necessary data to recreate a simple but at the same time very impressive version of the game.
Maybe what is shown in the video is not very sharp and is far from looking like the original GTA V; in fact, in the video it is explained that the original material is very pixelated and that a antialiasing very aggressive to soften the image and give it the final look.
But beyond that detail, this is only a first advance that makes us wonder about what these systems will be capable of in a few years.