NVIDIA's LATTE3D generative AI model creates game-ready 3D models and objects in seconds

NVIDIA showcased a new AI-powered technology that could revolutionize game development and the PC modding community. Described as a "virtual 3D printer," LATTE3D turns text prompts into 3D objects within seconds. The company notes that the 3D creations are created in a popular format that can be slotted into virtual environments for games or other applications with a few clicks.

"A year ago, it took an hour for AI models to generate 3D visuals of this quality - and the current state of the art is around 10 to 12 seconds," said Sanja Fidler, vice president of AI research at NVIDIA. "We can now produce results an order of magnitude faster, putting near-real-time text-to-3D generation within reach for creators across industries."

NVIDIA adds that by running on a single GPU like the NVIDIA RTX A6000, 3D shapes, animals, and objects are created instantly.

Like AI image generators, LATTE3D generates a few 3D models to give creators options. The video demonstration is impressive, with the AI creating accurate 3D objects for text prompts, such as a "sphynx cat on a skateboard" and a "cinnamon spice latte with cream." Naturally, the creations can be exported into NVIDIA Omniverse, which powers RTX Remix - adding a new tool for PC modders to transform classic games.

However, in its current form, LATTE3D is limited in what it creates because it was trained on two specific datasets on NVIDIA A100 Tensor Core GPUs: animals and everyday objects. The good news is that LATTE3D can generate models for various breeds of dogs or other animals so you won't get the same ones repeatedly.

NVIDIA notes, "Developers could use the same model architecture to train the AI on other data types," which is great to hear because a developer could train the AI using specific art. For example, Blizzard could train the AI model on everything related to Warcraft and potentially use LATTE3D to create background objects and scene decorations. That's just an example, with the idea being that generative AI could be used to help massive games and projects as a support tool for developers.