Sipeed's new K3 RISC-V SBCs can run 30B-parameter LLMs at 10 tokens per second

Sipeed has launched its new K3 series Single Board Computers, powered by the RISC-V ISA. Using SpacemiT's new "Fusion Architecture" with dedicated matrix multiplication blocks, Sipeed claims these systems can run 30B LLMs locally at over 10 tokens per second.

SpacemiT, a fabless Chinese semiconductor designer, is the silicon architect, while Sipeed serves as the hardware integrator. The K3 SoC is packaged with 32GB of LPDDR5-6400 memory and 10GbE networking, offering an alternative to proprietary AI hardware for enthusiasts and researchers. Prices start at $299 and go up to $629 for the 32GB flagship.

At the silicon level, the K3 SoC features 8 general-purpose X100 cores, each with 4 MB of L2 cache. The company claims the X100 performs similarly to ARM's Cortex-A76 core. The K3 also offers 8 A100 AI matrix units with TCM (Tightly Coupled Memory) and supports up to 1024-bit RVV 1.0 vector processing. These deliver up to 60 TOPS of performance (format unspecified) and support BF16, FP16, FP8, INT8, and INT4 data types.

3

Both X100 and A100 cores connect to the memory controller via a coherent interconnect bus, sharing the same memory. This is not like a traditional NPU, which usually resides separately. While not as fast as GPU cores, the K3 architecture supports zero-copy, allowing CPU and AI cores to share the same memory space.

The dual 32-bit controllers support LPDDR4x-4200/LPDDR5-6400 memory, delivering up to 51GB/s of bandwidth. The SoC is rated at a TDP of 15-25W. Sipeed offers the K3 in two form factors: the CoM (Computer-on-Module) and the Pico-ITX.

The K3 CoM260 is a 69.6mm x 45mm module, pin-compatible with NVIDIA's Jetson Orin Nano carrier boards. The Pico-ITX version, similar to a Raspberry Pi, measures 100mm x 86mm and features 2 USB Type-C ports (Power Delivery and Alt-DP), 1 10 GbE port, and 1 1 GbE port.

3

The K3 officially supports Ubuntu 26.04 and ROS. In terms of AI throughput, the company claims inference speeds exceeding 10 tokens per second with a 30B-parameter model. The 32GB version should be able to fit a quantized version of Qwen 3.6 A3B 35B (~22GB), though space will be tight. Smaller MoE models, such as Gemma 4 26B A4B (~15GB), may be more suitable.

The Sipeed K3 series, starting at $299 for the 8GB model, provides an accessible gateway into the RISC-V landscape. While it may not yet challenge NVIDIA's dominance in high-end GPUs, the K3 represents an important milestone as one of the first RVA23-compliant platforms with a full Ubuntu 26.04 LTS environment. The platform gives enthusiasts and researchers a practical way to explore local inference on an open instruction-set architecture.