Google's lightweight and free Gemma AI models are optimized to run on NVIDIA GPUs

"Built from the same research and technology used to create the Gemini models," Google's new lightweight Gemma LLMs have been designed to run natively on local PC hardware powered by NVIDIA GPUs.

2

Google's new Gemma AI is lightweight and runs natively on an AI PC, image credit: Google.

For use in third-party AI applications, Gemma 2B and Gemma 7B were both developed by Google DeepMind, with each open model capable of surpassing "significantly larger models on key benchmarks," according to Google, while running on a GeForce RTX-powered laptop or PC.

The Gemma pre-trained models are designed to be safe and reliable, with NVIDIA adding that the models are optimized to run on its open-source NVIDIA TensorRT-LLM library - accelerated by the over 100 million NVIDIA RTX GPUs available in AI PCs. Gemma can also run on JAX and PyTorch.

Plus, Gemma will also run on NVIDIA GPUs in the cloud. However, the goal appears to be a lightweight standalone AI model accessible by millions of PCs and laptops. You can test out Gemma 2B and Gemma 7B directly from your browser via the NVIDIA AI Playground by asking it a range of questions.

• Gemma 2B - NVIDIA AI Playground Demo.
• Gemma 7B - NVIDIA AI Playground Demo.

NVIDIA has also confirmed that Gemma is coming to its new Chat with RTX platform, an AI-powered chatbot that will run natively on RTX-powered Windows PCs running TensorRT-LLM software. Developers also have access to Gemma models, with Google setting up a hub with quick-start guides covering training and deployment.

The hub also includes some MMLU benchmark results showcasing Gemma 7B outperforming LLAMA-2 7B and 13B in problem-solving. For those who want a detailed look at Gemma, there's a detailed technical report from Google covering the model's architecture, training, and benchmark results.

"Beyond state-of-the-art performance measures on benchmark tasks, we are excited to see what new use cases arise from the community and what new capabilities emerge as we advance the field together," Google writes. "We hope that researchers use Gemma to accelerate a broad array of research, and we hope that developers create beneficial new applications, user experiences, and other functionality."