At Computex 2024, NVIDIA gave us a glimpse into the future of gaming with their impressive Digital Humans technology. This technology, developed in collaboration with Inworld AI and powered by NVIDIA ACE technologies, uses AI to create in-game characters with distinct personalities, knowledge, and relationships. These characters can be interacted with using text or voice, allowing players to engage in detective-style investigations and uncover hidden information. The demo showcased how this technology could be used to enhance gameplay, adding depth and interactivity to the gaming experience.
Now, at Gamescom 2024, NVIDIA is taking Digital Humans to the next level with their integration into Mecha BREAK, a multiplayer mech game developed by Amazing Seasun Games. This game utilizes NVIDIA’s new Nemotron-4 4B NIM small language model, which is specifically designed for game characters, enabling smoother and more realistic interactions with Digital Humans.
In Mecha BREAK, players can interact with a Digital Human named Martel, who provides insights about upcoming missions, mech customization options, and offers valuable advice. Martel can respond to voice commands, make suggestions, and even adjust mech configurations based on player input. This level of interactivity makes Martel an invaluable asset, especially for new players who might be navigating the game’s complex customization features.
To achieve this level of realism, NVIDIA utilizes various AI-powered technologies. Audio2Face-3D NIM creates dynamic facial animations based on the character’s speech. Speech recognition is handled by third-party models from Whisper, while Elevenlabs provides realistic voice generation in the cloud. The combination of these technologies creates a truly immersive and interactive experience.
While this cutting-edge technology requires powerful hardware to run locally, NVIDIA is also implementing cloud components to offload processing and minimize the impact on frame rate. Despite the demanding nature of Digital Humans, their integration into Mecha BREAK results in a negligible performance hit (around 5%) compared to the significant performance boosts offered by high-end GPUs.
As Digital Humans continue to evolve, they promise to revolutionize the gaming industry, offering new levels of immersion, interactivity, and storytelling. NVIDIA’s demonstration at Gamescom 2024 serves as a compelling example of how this technology can enhance the gaming experience and pave the way for a future where AI-powered characters become integral to the gaming landscape.
