Intel Nervana Neural Network Processor: 32GB HBM2 at 1TB/sec

Intel is hard at work on the research and development side of its upcoming Nervana Neural Network Processor, a new chip that will blow away any general-purpose processor for machine learning and AI applications.

2

Vice President of Hardware for Intel's Artificial Intelligence Products Group, Carey Kloss, has provided an update to the work Intel has made on the NNP.

What does a neural network processor (NNP) have to do? In order to train a machine using neural networks needs a gigantic amount of memory and arithmetic operations in order to generate useful output. Then we step into the scaling capabilities, power consumption and maximum utilization being the cornerstones of Intel's Nervana.

Intel want to keep power savings at a maximum, meaning Nervana is built to keep data within the chip and not into the system. Vector data is able to split between memory modules, keeping data local where it is super-fast.

2

This is where Intel's use of HBM2 comes into play with 32GB of HBM2 with over 1TB/sec of memory bandwidth on-tap with Nervana. The 1TB/sec is available between the on-die and external memory banks, providing a lot of bandwidth for neural network loads.

Intel wants more bandwidth but didn't want to wait for next generation memory technologies, landing on the best available now: HBM2. But, it wasn't good enough for Intel's Nervana platform so they made some adjustments.

There are modules of memory on Nervana of 2MB each totalling 30MB, with Intel using software to control the use of memory on-die to load information from external memory and then shifting data between the super-fast local memory modules.

This reduction in the reliance of external memory reduces the saturation of memory bandwidth, giving Intel the ability of pre-fetching the next data set required for operation.

Intel's next-gen Nervana NNP is the bleeding edge of tech, so as updates happen we'll keep you posted.