Artificial Intelligence - Page 40

In a recently shared data center AI chip roadmap posted on X, we get a good look at what companies have on the market already, and what's in the AI chip pipeline through to 2027. Check it out:

The list includes chip makers NVIDIA, AMD, Intel, Google, Amazon, Microsoft, Meta, ByteDance, and Huawei. You can see the list of NVIDIA AI GPUs includes the Ampere A100 through to the Hopper H100, GH200, H200 AI GPUs, and into the Blackwell B200A, B200 Ultra, GB200 Ultra and GB200A. But after that -- which we all know is coming -- is Rubin and Rubin Ultra, both rocking next-gen HBM4 memory.

We also have AMD's growing line of Instinct MI series AI accelerators, with the MI250X through to the new MI350 and the upcoming MI400 listed in there for 2026 and beyond.

Mistral-NeMo-Minitron 8B is a "miniaturized version" of the new highly accurate Mistral NeMo 12B AI model. It is tailor-made for GPU-accelerated data centers, the cloud, and high-end workstations with NVIDIA RTX hardware. Accuracy is often sacrificed to ensure performance regarding scalable AI models; Mistral AI and NVIDIA's new Mistral-NeMo-Minitron 8B deliver the best of both worlds.

Small enough to run in real-time on a workstation or desktop rig with a high-end GeForce RTX 40 Series graphics card, with NVIDIA, noting that the 8B or 8 billion variant excels when it comes to benchmarks for AI chatbots, virtual assistant, content generation, and educational tools.

Available and packaged as an NVIDIA NIM microservice (downloadable via Hugging Face), Mistral-NeMo-Minitron 8B is currently outperforming Llama 3.1 8B and Gemma 7B in the all-important accuracy category in at least nine popular benchmarks for AI language models.

NVIDIA will be hosting a Hot Chips Talk next week, deep diving into its new Blackwell GPU architecture while reminding the world that its Blackwell GPU has the highest AI compute, memory bandwidth, and interconnect bandwidth ever in a single GPU.

At Hot Chips 2024 next week, NVIDIA will go into more detail about the Blackwell GPU architecture while also reminding us that it features not one, but two reticle-limited GPUs merged into one. One of the limitations of lithographic chipmaking tools is that they've been designed to make ICs (integrated circuits) that are no bigger than around 800 square millimeters, which is referred to as the "reticle limit".

NVIDIA has two reticle-limited AI GPUs together (104 billion transistors per chip, 208 billion transistors in total). NVIDIA will discuss building to the reticle limit, and how it delivers on the highest communication density, lowest latency, and optimal energy efficiency during its Hot Chips Talk.

NVIDIA will discuss using AI to build next-generation chips for AI at the upcoming Hot Chips 2024 event next week.

The company will discuss how NVIDIA designs some of the most complex products on the planet, with its new Blackwell B200 AI GPU featuring 208 billion transistors, made on TSMC's new 4NP process node. At Hot Chips, NVIDIA will discuss using generative AI to generative optimized Verilog code.

What's Verilog code? Verilog is a hardware description language that describes circuits in the form of code. It's used for design and verification of processors, with NVIDIA building an Agentic AI LLM application to accelerate Computer Aided Engineering (CAE) that generates Verilog code, which can:

NVIDIA has powered up its new Blackwell AI GPUs and run them in real-time inside of their data centers, while teasing it will provide more details about its Blackwell GPU architecture at Hot Chips next week.

The company has been embroiled in rumors of its Blackwell AI GPUs having issues big enough to require a redesign, and issues with Blackwell AI servers leaking through their water-cooling setups. NVIDIA has now shown its new Blackwell AI GPUs running in real-time, with Blackwell on-track to ramp into production and ship (in small quantities) to customers in Q4 2024.

NVIDIA also teased new pictures of various trays available in the Blackwell family, with these first images of Blackwell trays being teased showing just how much engineering and design work goes into these things. It's truly incredible, almost like a work of art... and that's on the outside.

NVIDIA CEO Jensen Huang will be flying over to Taiwan to attend SEMICON Taiwan 2024, where he will meet with SK hynix and TSMC executives according to local media.

Jensen will reportedly meet with SK hynix's president at SEMICON, where the two will provide the world with an update on their progress, as well as working with TSMC on next-generation HBM memory chips like HBM4.

NVIDIA, TSMC and SK hynix recently formed a "triangular alliance" for a collaborative effort to make next-gen AI GPUs and next-gen HBM memory. SEMICON Taiwan 2024 is a big event for the semiconductor industry, and with the triangular alliance there, we should expect some updates to GPU and HBM roadmaps, and more.

South Korean telecommunications giant SK Telecom has announced its partnering with US-based GPU cloud startup Lambda on a new GPU farm using NVIDIA H100 AI GPUs.

The GPU farm filled with NVIDIA H100 AI GPUs will be owned by Lambda, and installed into the SK Broadband data center in Gasan-dong, Guro-gu, Seoul, by December this year. The new partnership on the GPU farm in South Korea is part of SKT's broader plan to expand its AI data center business reports Business Korea.

SKT plans to scale the number of AI GPUs to thousands in the next 3 years, with SK Broadband being a subsidiary of SKT, and will play a "crucial role" in this venture by providing optimized equipment management services to ensure the stable operation of the GPU servers.

NVIDIA's upcoming earnings report is ruffling some feathers already, one analyst saying Blackwell AI GPUs design flaws are seeing the B100 AI GPU "effectively canceled".

The design flaws plaguing NVIDIA's new Blackwell AI GPUs hit headlines a couple of weeks ago, where we began hearing about design flaws that analyst firm KeyBanc says NVIDIA will need to "respin" the Blackwell tile that will cause a 3-month delay on shipments.

KeyBanc explained: "Given the Blackwell delay, we believe NVIDIA will prioritize the ramp of B200 for hyperscalers and has effectively canceled B100, which will be replaced with a lower cost/performance GPU (B200A) targeted at enterprise customers".

SK hynix Vice President Ryu Seong-su has announced the South Korean AI memory leader is working on developing a product with 20-30x the performance of current-gen HBM memory.

During the recent "SK Icheon Forum 2024" event held at the Grand Walkerhill in Gwangjin-gu, Seoul, South Korea, the SK hynix VP said: "We aim to develop products with 20 to 30 times the performance of current HBM, focusing on introducing differentiated products".

Ryu emphasized SK hynix's focus on responding to the mass market, with AI memory solutions (HBM) through execution capabilities, reports Business Korea. This strategy by SK hynix has been crucial in the demand of high-performance memory to grow, driven by the insatiable demand for AI GPU hardware.

Samsung is rumored to tape-out its next-generation HBM4 memory in Q4 2024, with mass production of HBM4 expected by the end of 2025.

The AI industry is fueling the growth of HBM memory, with HBM makers SK hynix, Samsung, and Micron pumping out as much as it can, and it hasn't been enough. SK hynix is sold out of all of its HBM3 and HBM3E for both 2024 and 2025, and with HBM4 on the horizon, all things are leading to who can be the biggest HBM4 supplier to AI GPU companies like NVIDIA and AMD.

NVIDIA's next-generation Rubin R100 AI GPU uses HBM4 memory, so SK hynix and Samsung are both doing everything they can (spending many tens of billions of dollars, setting up and expanding semiconductor fabs in South Korea, Taiwan, and the United States) to pump more HBM memory (HBM3, HBM3E, HBM4, and even HBM4E) for 2025 and beyond.

AMD has announced it is acquiring server maker ZT Systems for $4.9 billion, in a bid to better compete in the AI industry against juggernaut NVIDIA.

The company plans to pay for 75% of the ZT Systems acquisition with cash, while the remainder will be provided in AMD stock. As of Q2 2024, AMD has $5.34 billion in cash and short-term investments at the ready, and ZT Systems is one of those investments.

ZT Systems is an artificial intelligence infrastructure group that will help AMD create a better AI infrastructure, helping get AMD's new Instinct MI300 series AI accelerators into data centers. ZT Systems was founded over 30 years ago, and is a private company that builds custom computing infrastructure for some of the world's biggest AI hyperscalers.

You probably haven't heard of Ola, but they're an Indian automotive manufacturer, which has just announced they're going to design, build, and launch India's first-ever in-house AI chip by 2026 powered by the Arm architecture.

India is now jumping into the AI semiconductor market, with its first-ever in-house AI chip aimed at self-driving vehicles of the future. AI inside of autonomous vehicles is a big business now, and will be even bigger in the future and India wants in on that market.

Ola CEO Bravish Aggrawal took the stage at an event recently, announcing multiple AI chips are coming from Ola, each specializing in their respective scenarios. Aggarwal underlined the importance of India creating AI chips in-house, instead of relying on third-party semiconductor companies like TSMC, Samsung, or Intel. The company showed off some of its Bodhi series of AI chips, including the Sarv-1-cloud-native CPUs and its new Ojas edge AI chip.

Geekbench AI 1.0 is here, after years of feedback and test iteration with its customers, partners, and the AI engineering community, Primate Labs is proud to announce its latest machine learning benchmark is now ready, and it has a new name: Geekbench AI. You can download Geekbench AI 1.0 here.

Geekbench AI is a new benchmarking suite with a testing methodology for machine learning, deep learning, and AI-centric workloads, all with the same cross-platform utility and real-world workload reflection that Primate Labs' benchmarking software (Geekbench, duh) is known for.

The developer explains on its website: "Measuring performance is, put simply, really hard. That's not because it's hard to run an arbitrary test, but because it's hard to determine which tests are the most important for the performance you want to measure - especially across different platforms, and particularly when everyone is doing things in subtly different ways. At Primate Labs, we build our tests to reflect the sort of use cases that developers build their applications to do through detailed and ongoing conversations with software and hardware engineers across the industry, rather than just crunching basic math for hours".

SoftBank was talking with Intel about making an AI chip that would compete with NVIDIA's dominant AI GPUs, but the plan "floundered" after Intel failed to meet the requirements of Japan's SoftBank.

In a new report from the Financial Times, we're hearing that negotiations to partner with Intel would've rapidly accelerated SoftBank's efforts to combine the chip design of its "crown jewel" as the FT puts it: Arm. Japan's SoftBank owns Arm Holdings, so with its Arm architecture and Intel fabbing the chip with SoftBank's production expertise of its latest acquisition -- Graphcore -- according to "people familiar with the matter".

SoftBank boss Masayoshi Son plans to dump billions of dollars into putting Japan at the center of the AI boom, where the Arm owner wants to create a rival to NVIDIA's dominant AI GPUs. Son has pitched his ideas to the usual Big Tech companies, with chip production and software through to providing power for the data centers that would be powered by its chips.

Samsung's new 8-layer HBM3E memory has passed NVIDIA's qualification tests to be used in its AI GPUs.

In a new report from Reuters, the outlet says that the qualification clears a "major hurdle" for Samsung -- which is the world's largest memory manufacturer -- and has been struggling to catch up to South Korean rival, SK hynix, which has been providing its bleeding-edge HBM3E memory to NVIDIA for its AI GPUs.

Samsung still hasn't signed a supply deal for the approved 8-layer HBM3E memory chips with NVIDIA, but "will do so soon" according to Reuters' sources, who declined to be identified as the matter remains confidential. The new 8-layer HBM3E memory ships from Samsung would be supplied to NVIDIA in Q4 2024, which is not far away now.

The world of AI PCs is just beginning with the first waves of Copilot+ ready PCs out in the wild, with 14% of PCs shipped globally in Q2 2024 featuring an NPU, making them an AI PC.

In a new report from Canalys, we're learning that 8.8 million AI-capable PCs were shipped in Q2 2024, with each of those 8.8 million systems featuring the latest AI processor (with an NPU) from AMD, Apple, Intel, and Qualcomm. As we see new generations of AI processors from AMD and Intel, Canalys says that we can expect rapid growth in the second half of this year, and going into high gear in 2025.

If we look at just the Windows segment, AI-capable PC shipments surged 127% sequentially in Q2 2024, with Lenovo shipping out its Qualcomm Snapdragon X Elite-based Copilot+ PCs with the Yoga Slim 7x and ThinkPad T14s, boosting its AI PC market share to around 6% of all Windows PC shipments, a mammoth 228% growth. HP followed behind Lenovo with a 7% share of AI PCs.

It looks like there is a "component shortage" for NVIDIA's new Blackwell-based GB200 AI servers, with market demand in Taiwan at the highest levels because of the AI market needing as many chips as possible.

In a new report from Taiwan Economic Daily, we're learning that NVIDIA's new GB200 AI servers are experiencing huge development issues, with the main problem stemming from leaks from the liquid cooling system inside of the next-generation multi-million-dollar AI server.

The water leakage inside of a GB200 AI server would be a disaster as you can imagine, with Taiwanese manufacturers moving into emergency mode to solve it, with new Taiwanese suppliers stepping up to help NVIDIA get its GB200 AI servers to market.

SK hynix has just announced it's planning to develop a 4F2 (square) DRAM, joining South Korean rival Samsung and its journey into the world of 3D DRAM.

The cost of EUV (extreme lithography) processes has continued to skyrocket since the commercialization of 1c DRAM, with SK hynix researcher Seo Jae Wook noted during an industry conference in Seoul, South Korea, on Monday.

The Elec reports that Seo said at the time whether manufacturing DRAM this way (using EUV) was profitable, where in response, SK hynix said it was considering manufacturing vertical gate (VG) or 3D DRAM for future DRAM. VG is what SK hynix internally calls 4F2, while Samsung calls theirs vertical channel transistor (VCT).

Last month we reported on AMD's announcement that it was acquiring the largest private AI lab in Europe - Silo AI. Today, AMD has followed up on the announcement to confirm that the "all-cash transaction valued at approximately $665 million" has now been completed, with Silo AI's scientists and engineers now a part of the AMD family.

$665 million in cash is nothing to sneeze at, and for AMD, the acquisition is the latest step in the company's broader pivot that puts its main focus on AI and AI-related technologies. This is nothing new; we've seen the same shift happen in other companies like Google, Meta, Apple, and, of course, NVIDIA. However, NVIDIA's AI focus started many years ago.

"AI is our number one strategic priority," said Vamsi Boppana, AMD senior vice president, AIG. "We continue to invest in both the talent and software capabilities to support our growing customer deployments and roadmaps."

GPU VRAM modding isn't new, but Chinese cloud companies are adding copious amounts of VRAM to the likes of the GeForce RTX 4090D (24GB modded to 48GB) and the RTX 4080 SUPER from 16GB to 32GB and renting them out at an hourly rate.

Now, we've got a Chinese AI expert that is talking with people in the country about selling NVIDIA's new GeForce RTX 4090D graphics card -- but with double the VRAM, from 24GB to 48GB of GDDR6X -- as well as the GeForce RTX 4080 SUPER upgraded from 16GB to 32GB of GDDR6X memory.

The source says that the GPUs are popular with cloud computing companies, as they're renting out the spare AI computing power because the demand is so high right now. The modded GeForce RTX 4080 SUPER with 32GB of VRAM is available for just $0.03 per hour, which is a damn good price for a monster modded RTX 4080 SUPER with 32GB of GDDR6X memory for AI workloads.