NEO Semiconductor's new 3D X-AI chip tech: replace HBM used in AI GPUs with 100x more perf

NEO Semiconductor has just unveiled the development of its new 3D X-AI chip technology, which aims to replace DRAM chips inside of HBM to solve data bus bottlenecks, by enabling AI processing through 3D DRAM.

2

The new 3D X-AI chip technology can reduce the huge amount of data transferred between HBM and GPUs during AI workloads, with NEO's innovative new 3D X-AI chip technology set to "revolutionize the performance, power consumption, and cost of AI chips for AI applications like generative AI".

NEO's new 3D X-AI chip technology has 100x the performance with 8000 neuron circuits to perform AI processing in 3D memory, a huge 99% power reduction that minimizes the requirement of transferring data to the GPU for calculation, reducing power consumption and heat generation by the data bus, and 8x the memory density with 300 memory layers, allowing HBM to store larger AI models.

A single 3D X-AI die has 300 layers of 3D DRAM cells with 128Gb capacity, and one layer of neural circuit with 8000 neurons. NEO's estimates that its new chip can support up to 10TB/sec of AI processing throughput per die, while using 12x 3D X-AI dies stacked with HBM packaging can achieve 120TB/sec processing throughput, which is a mind-boggling 100x performance increrase.

Andy Hsu, Founder & CEO of NEO Semiconductor, said: "Current AI Chips waste significant amounts of performance and power due to architectural and technological inefficiencies. The current AI Chip architecture stores data in HBM and relies on a GPU to perform all calculations. This separated data storage and data processing architecture makes the data bus an unavoidable performance bottleneck. Transferring huge amounts of data through the data bus causes limited performance and very high power consumption. 3D X-AI can perform AI processing in each HBM chip. This can drastically reduce the data transferred between HBM and GPU to improve performance and reduce power consumption dramatically".

Jay Kramer, President of Network Storage Advisors, said: "The application of 3D X-AI technology can accelerate the development of emerging AI use cases and promote the creation of new ones".