First digital human running locally on a GeForce RTX PC is here

NVIDIA ACE technology first debuted at Computex last year with a demo showcasing an AI-powered digital human or NPC that you could interact with. The first demo took place in a futuristic bar. With your microphone, you could walk up to the AI NPC and interact with them using natural language. It is like talking to a character in an RPG, but without a dialogue wheel, you decide, choose, and ask what you want.

2

The demonstration was running on NVIDIA's powerful GPU hardware in the cloud at the time - the sort of data center rig created to run complex and sophisticated AI models. We've seen the technology evolve and improve in the past year. Still, this year's Computex 2024 NVIDIA ACE 'Covert Protocol' game tech demo (developed in collaboration with Inworld AI) will run locally on a GeForce RTX AI PC.

NVIDIA ACE technologies like Audio2Face and NVIDIA Riva automatic speech recognition can now run locally on devices with GeForce RTX hardware.

NVIDIA Audio2Face is "a combination of artificial intelligence (AI)-based technologies that generate facial animation and lip sync driven only by an audio source." NVIDIA RIVA is "a set of GPU-accelerated multilingual speech and translation microservices for building fully customizable, real-time conversational AI pipelines. Riva includes automatic speech recognition (ASR), text-to-speech (TTS), and neural machine translation (NMT)."

The Covert Protocol tech demo made its debut at GTC earlier. It's built from a "digital human technologies suite that includes language, speech, animation, and graphics powered by AI." The tech demo uses RTX rendering technologies like real-time ray-tracing, RTGI, AI DLSS 3.5 Super Resolution, Ray Reconstruction, and Frame Generation.

Until now, the assumption was that the AI side of these digital humans would be run on cloud-based NVIDIA ACE servers, with GeForce RTX hardware handling all of the cutting-edge graphics, so it's impressive to see this now running locally. We'll check this out in person at Computex 2024, so stay tuned.