TinyCorp's new TinyBox AI system: AMD AI GPU system starts at $15K, NVIDIA GPU starts at $25K

TinyCorp has just announced that it will offer new TinyBox AI systems with either AMD or NVIDIA AI hardware inside. AMD systems will start at $15,000, while NVIDIA hardware inside will start at $25,000.

2

TinyCorp TinyBox AI system (source: TinyCorp)

The company offers its new TinyBox AI systems with 6 x AMD Radeon RX 7900 XTX graphics cards inside, starting at $15,000 per system. Meanwhile, TinyCorp offers a TinyBox AI system with 6 x NVIDIA GeForce RTX 4090 graphics cards, with this monster AI system starting at $25,000.

TinyCorp has been experiencing issues with AMD hardware, posting about its issues and new AI systems on X. The company explained that it was going to sell just AMD-powered systems, but due to various issues, it was forced to offer NVIDIA chips for AI GPU usage.

The company said that if you "like to tinker and feel pain" then choose the AMD option, where the "driver still crashes the GPU and hangs sometimes," adding that TinyCorp plans to "work together to improve it". TinyCorp said that they're going to start documenting the issues with the Radeon RX 7900 XTX GPU, trying to fix the issues and to expect an announcement from AMD in the future.

However, if you want it to "just work" then TinyCorp tells you to buy NVIDIA. The company adds that you "pay the tax" as in NVIDIA hardware is more expensive, but "it's rock solid," adding that there's "not too much more to say about it".

TinyCorp's new TinyBox AI system features up to 991 FP16 TFLOPs of compute performance from 6 x RTX 4090 graphics cards or up to 738 FP16 TFLOPs of compute performance from 6 x RX 7900 XTX graphics cards. There's up to 144GB of GPU RAM in total (with 6 x RTX 4090s, each of them have 24GB, so 24GB x 6 = 144GB).

There's up to 6.05TB/sec of memory bandwidth from the 6 x RTX 4090 graphics cards, or up to 5.76TB/sec of memory bandwidth from the 6 x RX 7900 XTX graphics cards. There's an incredible 28.7GB/sec of disk read bandwidth from storage, while the company is using a 32-core AMD EPYC processor.

On the power side of things, there's support for 2 x 1500W power supplies... more than enough power to keep 6 x RTX 4090s cranking out AI performance 24/7.

• 738 FP16 TFLOPs (AMD) / 991 FP16 TFLOPs (NVIDIA)
• 144 GB GPU RAM
• 5.76 TB/s (AMD) / 6.05 TB/s (NVIDIA) Ram Bandwidth
• 28.7 GB/s disk read bandwidth
• AMD EPYC CPU, 32 Cores
• 2x 1500W (two 120V Outlets, can power limit for less)
• Runs 70B FP16 LLaMA-2 out of the box using tinygrad
• $15,000 (AMD) / $25,000 (NVIDIA)

TinyCorp posted on X: "A hard to find"umr"repo has turned around the feasibility of the AMD tinybox. It will be a journey, but it gives us an ability to debug. We're going to sell both, red for $15k and green for $25k. When you realize your preorder you'll choose your color. Website has been updated. If you like to tinker and feel pain, buy red. The driver still crashes the GPU and hangs sometimes, but we can work together to improve it".

The post continued: "Going to start documenting the 7900XTX GPU, and we're going right to the kernel in tinygrad with a KFD backend. Also, expect an announcement from AMD, it's not everything we asked for, but it's a start. If you want"it just works"buy green. You pay the tax, but it's rock solid. Not too much more to say about it. Compare to more expensive 6x4090 boxes elsewhere. Hopefully we get both colors of tinybox on MLPerf in June".