NVIDIA DLSS Primer: Super Resolution, Frame Generation, and Ray Reconstruction explained

NVIDIA's DLSS or Deep Learning Supersampling suite of technologies and features now covers multiple generations of GeForce RTX hardware, advances, and upgrades. And to be honest, with DLSS 2 Super Resolution, DLSS 3 Frame Generation, and now DLSS 3.5 Ray Reconstruction, it can get a little confusing as to what does what - and what's supported by what hardware.

6

DLSS now covers a broad range of technologies for GeForce RTX hardware, image credit: NVIDIA.

What began as a way to upscale an image with the aid of AI to improve performance without adversely affecting image quality now extends to cover AI-based Frame Generation to improve perceived performance by up to 4X and Ray Reconstruction to make ray-traced lighting, reflections, and other effects look cleaner, sharper, and more realistic than what's possible with 'DLSS Off.'

Let's start at the beginning with DLSS 2 Super Resolution and how it makes use of the AI-based Tensor Cores in all GeForce RTX hardware - from the GeForce RTX 2060 up to the current flagship model, the GeForce RTX 4090.

DLSS 2 Super Resolution

6

DLSS has evolved over the years, image credit: NVIDIA.

Per the naming, DLSS 2 Super Resolution uses AI models that continue to learn to enhance and upscale a game's visual output to present a higher-resolution image without sacrificing image quality. And depending on the Quality preset (DLSS 2 offers Performance, Balanced, and Quality presets as a baseline), this can even deliver a cleaner and sharper image than rendering natively with TAA enabled.

DLSS 2 has evolved greatly over the years, with the latest version able to recreate finer detail and handle objects in motion without breaking a sweat. It's a technology that delivers excellent results whether you're gaming at 1080p, 1440p, or 4K - with the bonus of a sizable performance uplift that can offer up to 2X the performance in some cases.

DLSS 3 Frame Generation

6

Frame Generation offers a massive boost to performance for the GeForce RTX 40 Series, image credit: NVIDIA.

DLSS 3 was introduced last year alongside the new Ada Lovelace-powered GeForce RTX 40 Series of graphics cards, bringing the groundbreaking Frame Generation technology. This AI-based technology can generate entire frames between two frames, accurately rendering what the additional structure would look like. The result is a massive boost to frame rates in the region of 2X to 4X - and has resulted in games like Portal with RTX and Cyberpunk 2077's RT Overdrive Mode to present versions of games that would be unplayable without Frame Generation.

Frame Generation also leverages NVIDIA's latency reduction Reflex technology to ensure that games are still responsive when playing with DLSS 3 enabled. Since its debut NVIDIA has updated DLSS 3 to improve the quality of the AI-generated frames, and when implemented well, it can be a literal game-changer. DLSS 3 and Frame Generation make use of new AI "Optical Flow Accelerator" hardware exclusive to the GeForce RTX 40 Series - and it works across all models, from the mainstream GeForce RTX 4060 up to the enthusiast-class GeForce RTX 4080.

DLSS 3.5 Ray Reconstruction

6

A look at the DLSS 3.5 pipeline, image credit: NVIDIA.

Thanks to the new AI-based Ray Reconstruction, DLSS 3.5 makes ray-traced effects like reflections, shadows, and lighting look better. NVIDIA's latest supersampling technology uses an AI model trained on a tonne of ray-tracing data to replace hand-tuned denoisers to render pixels more accurately and remove visual artifacts. The result is more detail in reflections, fewer ghosting images, and a generally more realistic-looking ray-traced image.

"Shooting rays for every pixel on your screen is too computationally demanding, even for offline renderers that calculate scenes over the course of several minutes or hours," NVIDIA explains. "So instead, ray samples must be used - these fire a handful of rays at various points across the scene for a representative sample of the scene's lighting, reflectivity, and shadowing. The output is a noisy, speckled image with gaps, good enough to ascertain how the scene should look when ray-traced. To fill in the missing pixels that weren't ray-traced, hand-tuned denoisers use two different methods, temporally accumulating pixels across multiple frames and spatially interpolating them to blend neighboring pixels together. Through this process, the noisy raw output is converted into a ray-traced image."

Ray Reconstruction uses an NVIDIA supercomputer-trained AI network to generate higher-quality pixels in between sampled rays to make smarter decisions about using temporal and spatial data to generate the final image. The result is better RT. As the new addition to DLSS, AI-powered Ray Reconstruction will arrive first in intensive ray-tracing titles like Alan Wake 2, Cyberpunk 2077, and Portal with RTX. The good news is that the technology is supported by all GeForce RTX graphics cards.