Google's AI hardware push could challenge NVIDIA's dominance

Google could be preparing a major shake-up in the AI hardware space, with reports suggesting the company is working on new custom silicon designed to challenge NVIDIA's dominance.

2

According to a Reuters report, Google is in talks with Marvell Technology to develop two new chips to improve AI model efficiency. The first is a memory processing unit (MPU) designed to work alongside Google's existing Tensor Processing Units (TPUs), while the second is a next-generation TPU specifically optimized for AI inference workloads.

The MPU is particularly interesting because it represents a shift toward offloading memory-intensive tasks from the main accelerator, something that has become a major bottleneck in modern AI systems. By separating data movement and memory operations, Google could significantly improve performance and reduce latency.

Additionally, the new TPU design is expected to build on the company's existing accelerators, such as its current flagship platforms, which already deliver massive compute throughput and are deployed at scale across Google Cloud infrastructure. This move comes as Google continues to push its TPU ecosystem as a viable alternative to NVIDIA GPUs, which currently dominate the AI market.

With demand for AI compute skyrocketing, and supply constraints still a major issue, custom silicon tailored for inference workloads is becoming increasingly important, especially considering the rising prices of memory as a result of the tightening of supply.

While details remain limited and the talks are reportedly still ongoing, the potential partnership highlights Google's strategy: doubling down on vertically integrated AI hardware to gain greater control over performance, cost, and scalability in the rapidly evolving AI landscape, which will, in turn, enable Google to further increase the sophistication levels of AI.