How NVIDIA RTX AI technology is pushing PC gaming forward

AI or artificial intelligence has been a part of PC gaming for decades. Enemies that duck behind cover when taking fire, NPCs leaving their houses to work the field in The Elder Scrolls IV: Oblivion, pedestrians and cars moving out of the way in Grand Theft Auto: Vice City - it's all AI, more... or less.

We'd call these examples 'old AI,' where specific rules and a predetermined set of actions are followed, like someone completing a checklist. Modern AI or generative AI is different. Complex models are trained on large data sets that can generate, predict, and create content.

NVIDIA has been at the forefront of bringing this new form of AI to PC gaming. Since the arrival of the first GeForce RTX GPU in 2018, DLSS, or Deep Learning Super Sampling, has presented a new way to boost performance, giving PC gamers a generational improvement thanks to dedicated AI hardware called Tensor Cores.

As machine learning keeps evolving, so too has DLSS.

DLSS is now part of RTX AI, a suite of technologies that leverage the industry-leading AI capabilities of GeForce RTX graphics cards for developers, creators, and gamers. From video production to giving architects the power to showcase their creations with photo-realistic ray-traced lighting in real-time, this article will instead focus on the ways generative AI has changed the way we play games now, how we revisit games from the past and how it will change the future.

DLSS 3.5 Ray Reconstruction and Frame Generation have changed the game

One of the reasons DLSS was developed was that NVIDIA was at the forefront of another game changer, real-time ray tracing. Ray-tracing, or RT, requires a lot of GPU horsepower, making it the perfect candidate for cutting-edge AI technology. The jump from DLSS 1.0 to DLSS 3.5 is impressive. The AI-powered Super Resolution tech not only continues to deliver a sizable boost to performance but also maintains visual fidelity comparable to or better than native rendering.

7

In 2024, playing a PC game with ray-tracing, like Star Wars Outlaws, is an experience where DLSS is required. DLSS 3.5's Ray Reconstruction technology is, at its core, an AI solution for 'denoising,' a crucial step in the rendering process that makes the overall image clear and artifact-free. Ray Reconstruction isn't simply a replacement for existing denoisers; it dramatically improves the visual fidelity of ray-traced lighting, reflections, and more. In Star Wars Outlaws, it's night and day, DLSS 3.5 Ray Reconstruction makes the world come alive, and the shiny interiors of Imperial strongholds and Stormtrooper armor look crisp, detailed, and like the real thing.

Compared to the first PC games to implement ray tracing, the RT in Star Wars Outlaws is exponentially more complex and impossible to experience at its full potential without DLSS 3 and Frame Generation. Introduced with the GeForce RTX 40 Series, new specialized AI hardware and a powerful AI model are leveraged to generate entire frames. In Star Wars Outlaws, on a capable GeForce RTX 40 Series GPU, this means up to 100 FPS or more. Throw in the impressive latency reduction tech NVIDIA Reflex, another magic trick, and at any given point, more than half of the pixels you see on the screen are generated by AI. An incredible achievement in and of itself, but most importantly, it looks and feels great to play.

Digital Humans have arrived thanks to NVIDIA ACE technology

NVIDIA ACE isn't one technology but a suite of AI technologies that can be used to create Digital Humans with whom you can interact and engage in conversation. Partnering with AI visionaries like Convai and Inworld and game publishers like Ubisoft, these Digital Humans are already making their way to PC games. Running locally on a powerful GPU like the GeForce RTX 4090 or in the cloud on cutting-edge NVIDIA hardware, this technology is on track to change how we play forever.

7

How does it work? Well, it's far more complex, intricate, and involved than this description - but here goes. AI powers each Digital Human, and each Digital Human can be tailored and customized, aka given a personality. In NVIDIA's impressive Covert Protocol tech demo, each character has a history, a temperament, and specific things they know or don't know relative to the world.

Digital Humans respond dynamically when you speak to them using natural language, like engaging with a talking ChatGPT. Whether you're asking for the time, trying to locate something or someone crucial to the plot, or finding out more about their history, no response is scripted. AI generates everything. And by everything, NVIDIA ACE also uses generative AI to understand player speech, animate faces, and generate the emotive voice you hear. On top of this, DLSS Super Resolution, DLSS 3.5 Ray Reconstruction, and Frame Generation deliver an impressive ray-traced world to explore.

Imagining how this could work in a game like The Elder Scrolls: Skyrim or Cyberpunk 2077 is mind-boggling; talking to a party member in an RPG and getting a dynamic response would be a monumental moment for PC gaming. And it's right around the corner. The first game to include an NVIDIA ACE-powered Digital Human is Mecha BREAK, a multiplayer mech game where you can engage with the mechanic to get information on missions, load-outs, and more.

Project G-Assist is your AI digital assistant for PC gaming

NVIDIA's new Project G-Assist is an RTX-powered AI Assistant for PC games that can see what's on-screen while having immediate access to all the intricacies of the game you're playing and your PC hardware. That brief description might not sound all that revolutionary, but seeing is believing. To demonstrate the potential of Project G-Assist, NVIDIA partnered with the developers behind the Unreal Engine 5-powered ARK: Survival Ascended, a popular survival franchise set in a world full of dinosaurs, to showcase its potential.

7

Like DLSS, Project G-Assist is powered by a powerful Large Language Model (LLM) that understands game mechanics, rendering, and more. In ARK: Survival Ascended, you can ask it to identify what dinosaurs are roaming in front of you at any given moment. Bringing up your inventory, you can ask about various crafting recipes, what you can make, what you might need, and where to find it. Looking over your character stats, you can also get advice on how to spend skill points and what abilities will best suit your playstyle.

These walkthrough or Wiki-style aspects of Project G-Assist are impressive and feel like having an AI Assistant for PC games. However, it doesn't stop there. Project G-Assist can give you technical information, advice, and support.

7

Want to know how fast the game is running, presented as a graph covering the last five minutes of gameplay? Project G-Assist can whip that up for you in a few seconds. Want to optimize performance or GPU efficiency? Project G-Assist will help. With direct access to NVIDIA's in-depth performance metrics, it's like having direct access to someone who has spent days benchmarking and tweaking. It's like having one of the TweakTown staff on speed dial to optimize your game and rig.

RTX Remix uses AI to breathe new life into classic games

Digital Humans powered by NVIDIA ACE and Project G-Assist are new, incredible AI-powered technologies that will change the PC gaming landscape. However, there's one RTX AI technology from NVIDIA that isn't looking to the future - instead, it's looking to the past. Specifically, the rich history of PC gaming and the sizable and passionate community of developers and modders breathing new life into the games of yesteryear.

7

RTX Remix is a powerful and open-source tool that allows tinkerers and modders of all skill levels to take a classic game and inject it into RTX Remix. RTX Remix will automatically replicate and recreate the game with access to modern physically-based rendering and lighting.

Modders can then use RTX Remix to replace all lighting in older titles with full ray-tracing or Path Traced lighting. This feature alone is incredible. DLSS technologies like Super Resolution, Ray Reconstruction, and Frame Generation can also be leveraged to boost performance and fidelity. NVIDIA Reflex can improve system latency to make games feel smoother to play.

AI can be used to upscale and update existing textures to bring them into the world of 4K gaming. AI models like ComfyUI can generate brand-new textures and materials to change the look of the world you see. Models can be swapped out with more detailed versions to make a game from 2004 look like one from 2024. NVIDIA first showcased the potential of RTX Remix with its Portal with RTX mod, which looks so good that it feels like a full-blown remake of Valve's iconic first-person mind-bender.

7

Currently, tens of thousands of modders are experimenting with RTX Remix, and over a hundred or more RTX Remix projects are in development, covering everything from Deus Ex to Max Payne to Need For Speed. One of the most high-profile remixes in development is Half-Life 2: An RTX Remix Project from Orbifold Studios - a team of over 100 modders. With RTX Remix, one of the greatest PC games of all time, is being remade with some of the most incredible visuals. Technically, it's a mod, but the results are as impressive as what you'd get from a team of hundreds working for several years on a modern AAA game.