Meta's next-gen Llama3 LLM is here and the Intel Arc A770 outperforms the GeForce RTX 4060

Generative AI and overall LLM performance are set to become significant elements in measuring CPU and GPU performance in the coming years. Intel's latest deep dive into Arc graphics highlights LLM performance on its current flagship Intel Arc A770 16GB graphics card for desktops.

3

The GPU is generally available for around $300, and Intel is comparing its AI performance against NVIDIA's mainstream GeForce RTX 4060 8GB graphics card, which is its nearest Team Green price competitor. "We've looked at two discrete GPUs positioned in a similar price segment," Intel writes in its report.

Intel's test includes various models running with IPEX-LLM, an LLM library for PyTorch. This library contains optimizations for Intel hardware and uses the XMX AI acceleration found on Arc graphics cards to boost performance. On the GeForce RTX 4060 side, CUDA Torch is used.

With INT4 weight compression, FP16 execution, and a max output of 1024 tokens, the Intel Arc A770 16GB outclasses the GeForce RTX 4060 8GB when it comes to tokens-per-second performance. Overall, performance is up to 70% faster on the Arc A770 than the GeForce RTX 4060. The results also include the latest Llama3 model from Meta, which is cool. In fact, all Intel AI hardware supports Meta's next-generation Llama 3 LLM.

3

One reason for this is that many AI models are reliant on GPU memory capacity and speed. Thus, the faster 16GB of VRAM on the Intel Arc A770 is better suited to AI workloads than the slower 8GB of VRAM on the GeForce RTX 4060. Of course, optimizations and other factors account for discrepancies in performance, and NVIDIA's CUDA is still widely considered the gold standard.

Still, this is an impressive showing for Intel, and with the Intel Arc A770 16GB being one of the most affordable 16GB GPUs on the market by a wide margin, Intel's AI advancements make it an excellent option for those looking for a powerful and affordable AI solution.

• Read more: NVIDIA AI GPUs trained Meta's new Llama 3 model for the cloud, edge, and RTX PCs
• Read more: Meta launches early versions of Llama 3 to fight with OpenAI and Google in the AI battle
• Read more: Intel finishes assembly of ASML's first High-NA EUV tool, ready for Intel 14A process in 2025
• Read more: Intel Granite Rapids-AP CPU: LGA 7529 socket, up to 128 cores, up to 500W power