NVIDIA GeForce RTX 2070 Review: The Perfect Card For 1440p (Page 8)

Deep Learning Super Sampling

DLSS is some true new magic that NVIDIA has unleashed with the Turing GPU architecture, with Deep Learning Super Sampling being one of the most interesting parts of NGX, or Neural Graphics Acceleration. NGX is a new deep learning-based technology stack that is found inside of NVIDIA's new RTX technology.

NGX uses deep neural networks (or DNNs) as well as a set of Neural Services that will do AI-based functions that not only improve the graphics of supported games, but it also increases the performance. NGX uses Turing Tensor Cores for deep learning-based operations that end up with better-looking games and higher-performing tasks to users.

In most new games, rendered frames aren't displayed directly, they are pushed through post processing image enhancement that combines input from multiple rendered frames, where visual artifacts are hopefully removed, and aliasing hopefully stays in detail. Temporal Anti-Aliasing (TAA) is a shader-based algorithm that combines two frames using motion vectors to work out where to sample the previous frame, which is one of the most-used image enhancement tools used today. NVIDIA notes however, that the image enhancement process is very hard to get right.

This is where NVIDIA researchers thought that using AI to improve this task would come into play, with the company using its DNN (deep neural network) to solve this issue, which is where Deep Learning Super-Sampling (DLSS) was born. DLSS produces a much higher quality output image compared to TAA, with increased performance in some games that'll blow your socks off.

RTX 2080 Ti Is 100% Faster Than GTX 1080 Ti Thanks To DLSS

TAA will render its final target resolution and then combine frames, subtracting detail...compared to DLSS that renders much faster at a lower input sample count, then infers a result that at the target resolution, looks similar to TAA but does half the shading work to get there. In some tests, like the UE4 Infiltrator demo, there's a 100% performance improvement using the RTX 2080 Ti over the GTX 1080 Ti.

DLSS Feeds NVIDIA's Super Computer

One of the best parts of DLSS is that the results are great for the training process of the DNN, as it gets to stretch its legs on how to best product the desired output image based on large numbers of "super-high-quality" examples. NVIDIA trains its deep neural network by collecting thousands of what it calls "ground truth" reference images that are rendered with the "gold standard method for perfect image quality"... 64x super-sampling (64xSS).

64x super-sampling doesn't shade each pixel a single time, it shades at 64 different offsets within the pixel itself, combines the outputs, and products an image that is with "ideal detail and anti-aliasing quality". NVIDIA also captures matching raw input images rendered normally, where they begin training the DLSS network to match the output frames of 64xSS.

By going through each input, NVIDIA tells DLSS to produce an output and to then measure the distance between its output and the 64xSS image sample, where it "adjusts the weights in the network based on the differences, through a process called back propagation". DLSS will learn after many attempts and tests how to produce results that closely match 64xSS, while also learning how to avoid problems like blurring, dis-occlusion, and transparency that plagues TAA.

This is all under the standard DLSS mode but NVIDIA also has DLSS 2X which sees DLSS input being rendered at the final target resolution and then combined by an even bigger DLSS network that is capable of producing an output image that is close to the look of the 64x super-sampled rendering. NVIDIA notes that this would provide a result that would be "impossible to achieve in real-time by any traditional means".

DLSS Support - More On The Way

There are 15 games with DLSS support, and lots more of them on the way.

NGX Software Architecture

Another new thing inside of Turing is NGX or Neural Graphics Framework, which is where all of the AI models are created, stuffed into the driver and into your shiny new GeForce RTX graphics card.

NGX integration is easy: the AI model trains it, passes it through the neural network into the NGX API alongside the game engine, and into the rendered frame you see on your monitor.

We can expect some cool tricks from this, such as AI Super Rez that takes the image into the deep neural network and then blasts back a Super Res version of it. Imagine the possibilities with images, videos, and most of all gaming - powered by AI and DNNs.