NVIDIA ACE technologies are a game-changer in the world of digital humans. These AI-powered tools, previously showcased in the Covert Protocol tech demo, are capable of bringing digital humans to life with stunning realism. They generate personalities and dialogue, convert text to speech, and create lifelike facial animations, all powered by AI. For game developers and gamers, this means a whole new level of immersion and interaction in virtual worlds.
One of the most impressive tools in the NVIDIA ACE arsenal is Audio2Face-3D. This plugin seamlessly integrates with Autodesk Maya and Unreal Engine 5, using audio to generate incredibly accurate lip-syncing and facial animations for characters. NVIDIA even provides a sample Unreal Engine 5 project to guide developers in implementing ACE and creating their own Digital Humans.
According to NVIDIA’s Ike Nnoli, developers can build a database for their intellectual property, generate responses with minimal latency, and connect these responses to MetaHuman facial animations within Unreal Engine 5. These microservices are optimized to run on Windows PCs with low latency and a minimal memory footprint.
But accurate facial animation is just the tip of the iceberg. Retrieval-augmented generation (RAG) takes things a step further by allowing AI models and characters to understand conversational history and context. This means Digital Humans can engage in more natural and nuanced interactions with players, making them feel even more real.
The creation of these detailed Digital Humans, especially within Unreal Engine 5, requires significant AI processing power. To handle this demand, NVIDIA’s ACE technologies are designed to be cloud-based, offering developers a powerful platform for bringing their digital human creations to life.
