NVIDIA showed me its digital humans and it could be the most exciting gaming tech in 10 years

NVIDIA demoed me an updated version of its impressive ACE technology, which offers developers a suite of technologies designed to bring digital humans to life with generative AI models.

One of the biggest problems in gaming is the lackluster dialogue options, particularly in open-world or sandbox experiences that could significantly benefit from players being able to interact with NPCs more dynamically. NVIDIA ACE makes this possible, and while the technology is still in development what NVIDIA showcased to be was truly eye-opening at what is on the horizon for game's of the future.

NVIDIA ACE is designed to create digital humans in various forms, leveraging the following generative AI technologies: NVIDIA Riva ASR, TTS and NMT, which enables automatic speech recognition, text-to-speech conversation, and translation, NVIDIA Nemotron LLM for language understanding and contextual responses, NVIDIA Audio2Face for realistic facial animation based on audio tracks, and NVIDIA Omniverse RTX for real-time, path-traced realistic skin and hair.

How does this influence an NPC in a game? NVIDIA showed how a player could approach an NPC, ask it a question, and have that NPC produce a response appropriate to the question asked. The level of sophistication of the responses depended on a variety of factors, such as the guardrails the NPC AI had placed around its responses, if processing was occurring locally or was cloud-based, and how much "backstory" an NPC was given.

The level of responsiveness of the AI NPCs was incredibly impressive, and even with questions such as "describe the significance of your armor" resulted in unique, in-depth responses by the NPC. A prime example of how technology such as this will revolutionize the way gamers can interact with NPCs in games is the above video detailing Covert Protocol. Seeing this demo run in person made the penny drop for how valuable this technology will be for the various types of NPCs in games, and specifically what it means for gamers.

Picture and open-world game such as Grand Theft Auto VI and its thousands of NPCs having access to a dialogue system that enables players to speak to them verbally and then they reply dynamically to what is said. Additionally, this technology will lead to NPCs being able to remember conversations, which would result in NPCs remembering past actions of players and, thus, a completely unique experience for every player.

NVIDIA ACE is one of those technologies that isn't getting the attention it deserves, but once it's been adopted by a few major game releases, it will create a new standard for what gamers expect when it comes to interaction with NPCs. NVIDIA plans on baking this technology right into Unreal Engine 5 and making it as easy as possible for developers to take advantage of it.

• Read more: Half Life 2 RTX Remix hands-on - Hard to believe that it's a mod and not a remake