Microsoft has recently introduced Muse, an artificial intelligence model designed to generate both game visuals and gameplay actions. Developed in collaboration with Ninja Theory, a subsidiary of Xbox Game Studios, Muse aims to assist in game development by predicting game dynamics based on human input.

According to an official blog by Katja Hofmann, Senior Principal Research Manager and lead of the Microsoft Research Game Intelligence team, Muse is based on the World and Human Action Model (WHAM) and has been trained on more than one billion images and controller actions from the Xbox game 'Bleeding Edge', equating to over seven years of recorded gameplay. The model generates gameplay sequences by processing an initial one-second prompt and predicting subsequent frames and actions.

As per Microsoft, the company is already using Muse to develop real-time AI models for other first-party games. The research behind Muse, published in the science journal Nature, outlines its training process and capabilities. Microsoft has also made the model's weights, sample data, and a prototype interface available through Azure AI Foundry for further research and experimentation.

The development of Muse aligns with broader industry trends as video game production costs continue to rise. Microsoft and Ninja Theory have previously collaborated on AI research, with 'Bleeding Edge' serving as a platform for AI-driven gameplay experiments. Muse represents an extension of these efforts, focusing on the application of transformer-based models to human gameplay data, says Microsoft.