Switch 2's AI upscaling tech possibly detailed in new Nintendo patent

The United States Patent and Trademark Office has published a new Nintendo patent for AI-based image upscaling, giving us an idea of the kind of tech being used in the upcoming Switch 2 console.

4

Note: The following patent was published on December 31, 2024 in the United States, but was originally domestically filed on July 13, 2023 and published on November 9, 2023 in Japan.

It's 2025, and Nintendo's new Switch 2 console should be revealed soon. While gamers have been waiting, there's been a few Switch 2 leaks--some of which could be pretty huge if they turn out to be true. And just days ago, the USPTO published a very interesting new Nintendo patent that delves into how the company is using AI upscaling technology in the Switch 2 hardware.

The Switch 2 is believed to use an 8nm NVIDIA Orin-based SoC that's been customized for handheld gaming. The Orin chips are typically used for smart automobiles, but have unique power-to-performance profiles that can be scaled for on-the-go handheld gaming. NVIDIA's Orin SoCs come with built-in Tensor cores that facilitate DLSS image upscaling; conversely, the current-gen Switch console uses a Tegra X1 SoC that is Tensor-less, and like Orin, the Switch's Tegra X1 chip was also used for NVIDIA's early Drive PX autonomous vehicle platform.

The new patent (SYSTEMS AND METHODS FOR MACHINE LEARNED IMAGE CONVERSION - United States Patent 12182966 - B2) goes into significant amounts of detail on the topics of machine learning and AI upscaling, including all of its forms, the different neural networks (Convolutional Neural Networks, Recurrent Neural Networks, etc. - if you saw the Mark Cerny PS5 Pro tech talk, you'll recognize these), and how the images are transformed via matrices.

4

The patent mentions that the hardware will have a special dedicated hardware register and corresponding processing core that will handle the matrix-based image transformations.

One interesting part is how the patent describes using a virtual camera to take a snapshot of any given scene in the game. This snapshot is then sent to and manipulated by the neural network, transforming it through the matrix-based input-output method, and then beaming the image to an output screen.

Another noteworthy point: The patent frequently underlines the impact of image upscaling.

Throughout the text, the authors make it clear that developers can significantly reduce the hardware demands of their games by shrinking the native image output of their games engines. Instead of a 4K image, devs can use a 1080p image and blow it up using the techniques exhaustively described within the patent. Or, instead of a 1080p image, they can use a 540p image with native low-res textures and transform the image to better fit smaller screens.

4

Another interesting thing to note: Nintendo has been investing in AI and machine learning for a while now. Since 2019, to be exact.

Every year, Nintendo spend hundreds of millions of dollars on Research and Development, but this includes the totality of its operations: Hardware, game development/software, services, etc. It's unknown how much Nintendo has spent on investigating AI-based upscaling.

If the source link to the patent doesn't work, type 12182966 into the search bar at this link here.