Studios like Tencent and Ubisoft are using NVIDIA ACE to create AI NPCs in upcoming games

NVIDIA first showcased its new NVIDIA Avatar Cloud Engine (ACE) technology created in partnership with Convai back at Computex 2023, with a Cyberpunk-style diner showcasing a fully realized AI character you can interact with (via voice) who will then dynamically respond and react depending on what's happening in the game world.

At CES 2024, NVIDIA is presenting an updated version of the diner demo with an additional character in the scene and confirmation that a broad list of developers are now using the technology for game development. Charisma.AI, Convai, Inworld, miHoYo, NetEase, Games, Ourpalm, Tencent, Ubisoft and UneeQ.

Tencent calls NVIDIA ACE a "milestone moment for AI in games" and will use the tech in upcoming projects. NVIDIA notes that the latest version gives AI NPCs awareness of objects so they can pick them up and drop them off. In addition, NPCs can lead players.

Here's an overview and a look at how NVIDIA ACE integrates with the cloud and game engines.

4

With NVIDIA ACE, the player's voice is converted to text and put through cloud-based generative AI before being sent back to the game engine.

4

Facial animation is generated on-the-fly with Audio2Face.

4

NVIDIA ACE in action.

Granted, following an NPC to a quest marker isn't exactly the high-watermark of game design, but AI interaction like this could lead to some truly immersive open-world environments. NPCs that dynamically change and react to your actions and events would be a game changer.

ACE is a full suite that lets developers build interactive avatars. NVIDIA Audio2Face creates facial animations to match the audio services with NVIDIA Riva using generative AI to offer multilingual speech and translation. Audio2Face and Riva Automatic Speech Recognition microservices are available now.

"Generative AI-powered characters in virtual worlds unlock various use cases and experiences that were previously impossible," said Purnendu Mukherjee, founder and CEO at Convai. "Convai is leveraging Riva ASR and A2F to enable lifelike NPCs with low-latency response times and high-fidelity natural animation."

"Generative AI technologies are transforming virtually everything we do, and that also includes game creation and gameplay," said Keita Iida, vice president of developer relations at NVIDIA. "NVIDIA ACE opens up new possibilities for game developers by populating their worlds with lifelike digital characters while removing the need for pre-scripted dialogue, delivering greater in-game immersion."

The use of AI for art and other applications that thrive on human creativity is controversial, though this feels like an additive. Using NVIDIA ACE to help flesh out a world, like procedural generation and existing 'simple AI' routines for NPCs, is genuinely exciting.

Here's hoping NVIDIA releases a playable demo for this diner scene - it would be super interesting to test out the tech first-hand.

• Read more: GeForce RTX 4070, RTX 4070 Ti, and RTX 4080 SUPER announced - pricing, specs, and performance!
• Read more: GeForce NOW is getting new Blizzard games, Day Passes, and G-SYNC technology
• Read more: GeForce RTX 4070 SUPER and RTX 4080 SUPER Founders Edition models look incredible in all black
• Read more: Twitch Enhanced Broadcasting allows for five concurrent streams with a single GeForce RTX GPU
• Read more: GeForce RTX 4070 SUPER Founders Edition unboxing