Google's new artificial intelligence turns text into 3D objects

Google originally unveiled its generative 3D AI system called Dream Fields in 2021, and now a new and improved version has arrived.

Google's new next-generation artificial intelligence software designed to convert text into 3D generated images is called DreamFusion. So, how does this work? In a new proof-of-concept paper published to the pre-print server arXiv, researchers outlined that Dream Fusion, much like Dream Fields, uses a neural network called Neural Radiance Field (NeRF) that is designed to general novel views of complex 3D scenes using 2D datasets.

However, DreamFusion has taken a different approach than Dream Fields, as explained by Google research scientist Ben Poole who wrote on Twitter that the team replaced OpenAI's CLIP technology that powered Dream Fields with Google's own AI model called Imagen. The 3D models seen above and below aren't as photo-realistic as what we've seen with Midjourney. However, they are certainly still impressive as the 3D models have accurate surface geometry, depth and are even relightable for various lighting conditions.

Furthermore, Poole explains on Twitter that several 3D models generated by DreamFusion can even be thrown into a single scene.

"We're excited to incorporate our methods with open source models and enable a new future for 3D generation!" wrote Poole.

"Our approach requires no 3D training data and no modifications to the image diffusion model, demonstrating the effectiveness of pretrained image diffusion models as priors," Google's research team writes.

If you are interested in checking out the 3D models for yourself, head on over to the DreamFusion website here.