AI experts at OpenAI developed and trained a neural network to play Minecraft as well as humans. They trained the AI for 70,000 hours on miscellaneous in-game footage. In addition, experts even included a small database of videos showing contractors performing specific in-game tasks for the training. On top of that, they also included the keyboard and mouse inputs recorded.
Furthermore, the OpenAI experts soon found that the model could perform all sorts of complex skills. For instance, swimming, hunting for animals, consuming meat, etc. Additionally, the AI system even performed the “pillar jump.” It is a mover where players place a block of material below themselves mid-jump for more elevation. Besides that, it could also craft diamond tools that require a long string of actions executed in sequence. The experts at OpenAI stated this ability of the model as an “unprecedented” achievement for a computer agent.
Importance of an AI system playing Minecraft
The Minecraft project by OpenAI demonstrates the company’s new technique to train AI models. It is called Video PreTraining (VPT), and according to OpenAI, it could accelerate the development of “general computer-using agents.”
Previously, using raw videos for training AI models was quite tricky. For instance, an AI model would absorb the desired outcomes of a game. But, it wouldn’t be able to grasp the input combinations needed to reach those outcomes.
However, VPT overcomes that issue. With this technique, OpenAI pairs a large video dataset from public web sources and a carefully curated pool of footage. Next, the footage is labeled with relevant keyboard and mouse movements. And the team at OpenAI uses all of these to establish the model’s foundation. After that, the team fine-tunes the base model by plugging in smaller datasets designed to teach specific tasks.
For the Minecraft project, OpenAI used footage of players performing early-game actions. They found that doing so was a “massive improvement” that proved the model’s reliability in performing these tasks. Moreover, the OpenAI experts also rewarded the AI model for achieving each step in a sequence of tasks. It is similar to reinforcement learning used for children.
OpenAI states, “VPT offers exciting possibilities of directly learning large-scale behavioral priors in domains other than language.” The experts believe that the results from the Minecraft project suggest success for other similar domains like computer usage.
Lastly, to encourage further experiments in the space, OpenAI partnered with the MineRL NeurIPS competition with a grand prize of $100,000. Consequently, they donated their contractor data and model code to the contestants attempting to use AI to solve complex Minecraft tasks.