Ubisoft's Project NEO NPC game demo uses NVIDIA's Audio2Face and Inworld generative AI

Project NEO NPC in a new game demo and prototype from Ubisoft that goes all in on generative AI to create believable characters you can talk to.

2 minutes & 55 seconds read time

Yesterday, we examined AI's use to create digital human characters (NPCs) as part of a new Covert Protocol tech demo from Inworld and NVIDIA. Using the Inworld AI engine with NVIDIA ACE, which includes NVIDIA Riva Automatic Speech Recognition (ASR) and NVIDIA Audio2Face (A2F) for speech and animation, the demo gave a glimpse at a potential AI-powered future for games.

Ubisoft's generative AI Project NEO NPC game demo, image credit: Ubisoft.

Ubisoft's generative AI Project NEO NPC game demo, image credit: Ubisoft.

At GDC 2024, Ubisoft and its developers at Ubisoft Paris presented their demo, Project NEO NPC, in partnership with NVIDIA and Inworld. Using Inworld's extensive Character Engine with LLM technology, the team created NPCs with complete backgrounds, knowledge, and distinct conversational styles Ubisoft leveraging NVIDIA Audio2Face.

"Characters are programmed to react to player prompts in certain ways, or not react, depending on their personalities," Ubisoft writes in its blog post. Narrative Director Virginie Mosser at Ubisoft Paris adds, "For the first time in my life, I can have a conversation with a character I've created. I've dreamed of that since I was a kid."

Based on the screenshots, Project NEO NPC's story involves a heist with players' given objectives and goals.

Ubisoft adds that Project NEO NPC is a prototype, and its veteran designers had to iterate to ensure that responses were not "robotic" or "boring." Getting its AI-driven character to react like real people or fictional characters with specific traits and motivations was a collaborative process between creators, engineers, and AI specialists.

Ubisoft's generative AI Project NEO NPC game demo, image credit: Ubisoft.

Ubisoft's generative AI Project NEO NPC game demo, image credit: Ubisoft.

"It is really important to us that it behaves like the character Virginie created," said Data Scientist Mélanie Lopez Malet. "So, while we're talking to it, we ask ourselves: 'Is this Lisa? Would Lisa say this?' and if the answer is no, we need to go back and find out what happened within the model to make it stray from the vision Virginie had."

"It's garbage in, garbage out," Malet adds when players input 'nonsensical prompts.' "But it's also magic in, magic out. And when the player creates their own scene and it all clicks? That's an incredible experience for them."

The model also has measures to account for toxicity or players insulting characters, so if you actively try to break the fourth wall, the system won't work as intended - which is to be a collaborative experience for players. Ubisoft believes generative AI and storytelling "can live side by side," but there's "still a way to go before it can be implemented in a game."

Buy at Amazon

ZOTAC GeForce RTX 4070 SUPER Twin Edge Graphics Card

TodayYesterday7 days ago30 days ago
Buy at Newegg
* Prices last scanned on 4/24/2024 at 1:27 pm CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.
NEWS SOURCE:news.ubisoft.com

Kosta is a veteran gaming journalist that cut his teeth on well-respected Aussie publications like PC PowerPlay and HYPER back when articles were printed on paper. A lifelong gamer since the 8-bit Nintendo era, it was the CD-ROM-powered 90s that cemented his love for all things games and technology. From point-and-click adventure games to RTS games with full-motion video cut-scenes and FPS titles referred to as Doom clones. Genres he still loves to this day. Kosta is also a musician, releasing dreamy electronic jams under the name Kbit.

Newsletter Subscription

Related Tags