Train and Run Your Own AI with GIGABYTE AI TOP

Like all PC technology breakthroughs, generative AI combines cutting-edge hardware with the right software, tools, and data. In today's AI PC era, many AI models are general-purpose or trained on a specific data set, running on the cloud with powerful hardware buried in a large data center.

Conducting AI training locally with security in mind is critical for individuals, groups, or businesses concerned with security and broadcasting sensitive data over a network. For some, it's also a question of where to begin - with the understanding that you'll need some serious hardware.

Thankfully, GIGABYTE has created AI TOP - a combination of AI tools and hardware designed to make training and running your powerful AI at home or on a desk a reality with off-the-shelf GIGABYTE hardware and software.

GIGABYTE's new AI TOP Utility is a tool built to provide a robust, user-friendly environment for local AI model training and customization for all skill sets. It supports the most common open-source LLMs or large language models and can use custom datasets. It's the ideal solution for businesses, organizations, or even individuals looking to tap into cutting-edge tech.

10

Regarding AI, parameters are the various variables learned through the training process. They are often used as a metric to define the AI model's overall skill, accuracy, and performance. The more parameters, the better, and GIGABYTE AI TOP supports up to 236B-parameter large language models or 236 Billion. Until now, training an AI model of this size would have required tapping into the power of a data center built for generative AI training.

The AI TOP Utility pairs with GIGABYTE AI TOP hardware to ensure there is enough GPU VRAM, motherboard, PSU, and SSD power for local AI training and fine-tuning. One of the most impressive aspects of GIGABYTE's new AI TOP Utility and hardware is that you don't need extensive AI programming knowledge, or any, before jumping in.

Introducing the AI TOP Utility

10

Traditional AI training methods can be cumbersome and complicated. GIGABYTE's AI Top Utility for Linux systems has been designed as a single interface and tool that can handle AI training and fine-tuning with real-time hardware monitoring and logging tools for troubleshooting and customization. The AI Top Utility supports over 70 open-source LLM models (via Hugging Face), including Gemma 2, Llama 2, Llama 3, Baichuan 2, Distill-GPT2, and GLM4.

AI Top Utility offers a clean and easy-to-follow interface. At its most basic level, you can create a new Experiment by selecting the AI model, defining the Dataset, and choosing whether you want speed, precision, or a balance between the two. You might think precision is the way to go, but results will take longer to generate, and a longer training time is involved.

From there, you can drill into various training settings for more advanced options to control how the AI models behave and perform during training. GIGABYTE offers detailed breakdowns of each setting, including guides on creating AI datasets as part of the AI Top Utility manual.

10

You've also got control over what GPU hardware and memory to utilize for training, with the ability to combine VRAM with DRAM and SSD storage for the most memory-intensive training.

One of the critical features of the AI TOP Utility is the Dashboard, which offers the current progress of training and stats on system performance. From GPU and CPU measurements presented in an easy-to-read odometer fashion to a real-time look at VRAM, DRAM, and SSD Loading, you can get an immediate and clear picture of how your AI TOP system is performing.

The Dashboard also provides a real-time look at the recorded logs while tracking how effectively the model learns from the dataset. The information provides a quick overview of how things are progressing or potentially stalling, depending on the configuration and hardware available.

GIGABYTE's powerful AI TOP Hardware

10

The innovative and groundbreaking AI TOP Utility is designed to work specifically with GIGABYTE's powerful AI TOP hardware, starting with the GIGABYTE TRX50 AI TOP Motherboard. GIGABYTE also offers AI TOP GPUs, SSDs, and a PSU - all created with high-end AI performance in mind.

10

As for the TRX50 AI TOP Motherboard, this is the beast you'll need to run powerful AI models with the AI TOP Utility. It comes equipped with 4 x PCIe 5.0 x16 slots to house up to four GPUs. In addition, you've got 8 Channel DDR5, with support for up to 2TB of system memory. And just to make it even more impressive, its Socket sTR5 supports high-end AMD Ryzen Threadripper 7000 and Threadripper PRO 7000 Series CPUs.

Storage-wise, it can also house up to 4 PCIe Gen5 SSDs, features USB4 support, a Dual 10 GbE LAN for high-speed networking, and Wi-Fi 7. To ensure you get the best AI performance, the TRX50 AI TOP Motherboard also features a 16+8+4 Phases Digital VRM Solution for power delivery, thermal guards, and the same DIY-friendly scalability and GIGABYTE features like EZ-Plug and EZ-Latch that powers the company's most celebrated motherboards.

10

For those looking to install more than one GPU into their AI TOP-powered PC, GIGABYTE offers a range of AI TOP Graphics Cards, including the GeForce RTX 4070 Ti SUPER AI TOP 16G, Radeon PRO W7900 Dual Slot AI TOP 48G, and Radeon PRO W7800 AI TOP 32G.

These are all AI-ready because they feature the company's Turbo Fan cooling, which is designed to maximize airflow for multiple cards and compact dual-slot sizes with metal back plates. Running two or more GPUs like this doubles, triples, or quadruples your VRAM capacity and potential for running more complex AI training. VRAM capacity is extremely important for large-scale AI training.

10

GIGABYTE's AI TOP 100E SSD is available in 1TB or 2TB variants and has been designed for intensive AI workloads where TBW (Terabytes Written), stability, and an expanded memory pool are essential in maintaining accuracy and performance. With a complete AI TOP system, the goal is to leverage GPU VRAM, System RAM, and an AI TOP SSD like this to offload the processing of large datasets to the SSD, which not only saves time but also overall costs.

With the sheer amount of data involved in AI training, the AI TOP 100E SSD has a 150 times higher TBW rating than standard SSDs.

10

With a Threadripper CPU, up to four GPUs, eight channels of DDR5 memory, and multiple M.2 SSDs all firing simultaneously, the final piece of the hardware puzzle is the PSU or power supply. GIGABYTE's UD1600PM PG5 AI TOP is a powerful 1600W PSU with two PCIe 16pin to PCIe 16pin connectors with additional 8pin to PCIe 16pin connectors to support up to four PCIe Gen 5.0 graphics cards. Each card received gets its own dedicated 300W power supply.

With a fully modular design and an 80 PLUS Platinum certification, the UD1600PM PG5 AI TOP supports up to 200% power excursions and is described as 'sever-grade' and ultra-durable.'

For more, there's GIGABYTE's AI TOP Tutor

10

As you can see, GIGABYTE AI TOP is a comprehensive software and hardware solution that allows anyone to set up, train, and run complex AI models locally with complete control over datasets, parameters, and fine-tuning. However, one final piece is the AI TOP Tutor, a generative AI model created by GIGABYTE that can provide consultations, guidance for setting up hardware and software, optimizing hardware and training, and even technical support.

Interfacing with AI TOP Tutor is done using natural language and a chat interface. To bring everything full circle, the AI TOP Tutor was trained using the AI TOP Utility and AI TOP Hardware outlined in this article.

Head here for more on GIGABYTE AI TOP.