TT Show Episode 58 - Apple Intelligence, AMD confirms RDNA 4 details, and Russia fines Google

Lightweight AI - NVIDIA releases Small Language Model with industry leading accuracy

Mistral-NeMo-Minitron 8B is a highly accurate and powerful small language model built off of NVIDIA and Mistral AI's NeMo 12B, optimized for real-time.

Lightweight AI - NVIDIA releases Small Language Model with industry leading accuracy
Comment IconFacebook IconX IconReddit Icon
Senior Editor
Published
1 minute & 15 seconds read time

Mistral-NeMo-Minitron 8B is a "miniaturized version" of the new highly accurate Mistral NeMo 12B AI model. It is tailor-made for GPU-accelerated data centers, the cloud, and high-end workstations with NVIDIA RTX hardware. Accuracy is often sacrificed to ensure performance regarding scalable AI models; Mistral AI and NVIDIA's new Mistral-NeMo-Minitron 8B deliver the best of both worlds.

Lightweight AI - NVIDIA releases Small Language Model with industry leading accuracy 2

Small enough to run in real-time on a workstation or desktop rig with a high-end GeForce RTX 40 Series graphics card, with NVIDIA, noting that the 8B or 8 billion variant excels when it comes to benchmarks for AI chatbots, virtual assistant, content generation, and educational tools.

Available and packaged as an NVIDIA NIM microservice (downloadable via Hugging Face), Mistral-NeMo-Minitron 8B is currently outperforming Llama 3.1 8B and Gemma 7B in the all-important accuracy category in at least nine popular benchmarks for AI language models.

"We combined two different AI optimization methods - pruning to shrink Mistral NeMo's 12 billion parameters into 8 billion, and distillation to improve accuracy," said Bryan Catanzaro, vice president of applied deep learning research at NVIDIA. "By doing so, Mistral-NeMo-Minitron 8B delivers comparable accuracy to the original model at lower computational cost."

Pruning and distillation for AI training involves downsizing the neural network by removing components that "contribute the least to accuracy" and retraining the pruned model via distillation. NVIDIA has also confirmed that it has an even "smaller" version called Nemotron-Mini-4B-Instruct, which is optimized for low memory and faster response times on NVIDIA GeForce RTX AI PCs and laptops.

For more information on Mistral-NeMo-Minitron 8B, check out NVIDIA's technical blog.

Photo of the PlayStation 5 Slim Disc console
Best Deals: PlayStation 5 Slim Disc console
Country flag Today 7 days ago 30 days ago
Loading... Loading...
Buy
$499.99 USD -
Buy
* Prices last scanned on 11/2/2024 at 4:55 am CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission from any sales.
Newsletter Subscription

Join the daily TweakTown Newsletter for a special insider look into new content and what is happening behind the scenes.

Senior Editor

Email IconX IconLinkedIn Icon

Kosta is a veteran gaming journalist that cut his teeth on well-respected Aussie publications like PC PowerPlay and HYPER back when articles were printed on paper. A lifelong gamer since the 8-bit Nintendo era, it was the CD-ROM-powered 90s that cemented his love for all things games and technology. From point-and-click adventure games to RTS games with full-motion video cut-scenes and FPS titles referred to as Doom clones. Genres he still loves to this day. Kosta is also a musician, releasing dreamy electronic jams under the name Kbit.

Related Topics

Newsletter Subscription