Google's lightweight and free Gemma AI models are optimized to run on NVIDIA GPUs

Lightweight enough to run on an RTX AI PC or laptop, Google's new Gemma open models were built with the same technology used to create the Gemini models.

Published
Updated
2 minutes & 4 seconds read time

"Built from the same research and technology used to create the Gemini models," Google's new lightweight Gemma LLMs have been designed to run natively on local PC hardware powered by NVIDIA GPUs.

Google's new Gemma AI is lightweight and runs natively on an AI PC, image credit: Google.

Google's new Gemma AI is lightweight and runs natively on an AI PC, image credit: Google.

For use in third-party AI applications, Gemma 2B and Gemma 7B were both developed by Google DeepMind, with each open model capable of surpassing "significantly larger models on key benchmarks," according to Google, while running on a GeForce RTX-powered laptop or PC.

The Gemma pre-trained models are designed to be safe and reliable, with NVIDIA adding that the models are optimized to run on its open-source NVIDIA TensorRT-LLM library - accelerated by the over 100 million NVIDIA RTX GPUs available in AI PCs. Gemma can also run on JAX and PyTorch.

Plus, Gemma will also run on NVIDIA GPUs in the cloud. However, the goal appears to be a lightweight standalone AI model accessible by millions of PCs and laptops. You can test out Gemma 2B and Gemma 7B directly from your browser via the NVIDIA AI Playground by asking it a range of questions.

NVIDIA has also confirmed that Gemma is coming to its new Chat with RTX platform, an AI-powered chatbot that will run natively on RTX-powered Windows PCs running TensorRT-LLM software. Developers also have access to Gemma models, with Google setting up a hub with quick-start guides covering training and deployment.

The hub also includes some MMLU benchmark results showcasing Gemma 7B outperforming LLAMA-2 7B and 13B in problem-solving. For those who want a detailed look at Gemma, there's a detailed technical report from Google covering the model's architecture, training, and benchmark results.

"Beyond state-of-the-art performance measures on benchmark tasks, we are excited to see what new use cases arise from the community and what new capabilities emerge as we advance the field together," Google writes. "We hope that researchers use Gemma to accelerate a broad array of research, and we hope that developers create beneficial new applications, user experiences, and other functionality."

Buy at Amazon

ASUS Dual GeForce RTX 4070 SUPER OC Edition

TodayYesterday7 days ago30 days ago
$609.00$609.00$599.99
Buy at Newegg
$619.99$609.99$599.99
* Prices last scanned on 4/24/2024 at 11:06 am CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.

Kosta is a veteran gaming journalist that cut his teeth on well-respected Aussie publications like PC PowerPlay and HYPER back when articles were printed on paper. A lifelong gamer since the 8-bit Nintendo era, it was the CD-ROM-powered 90s that cemented his love for all things games and technology. From point-and-click adventure games to RTS games with full-motion video cut-scenes and FPS titles referred to as Doom clones. Genres he still loves to this day. Kosta is also a musician, releasing dreamy electronic jams under the name Kbit.

Newsletter Subscription

Related Tags