Artificial Intelligence - Page 39

NVIDIA has powered up its new Blackwell AI GPUs and run them in real-time inside of their data centers, while teasing it will provide more details about its Blackwell GPU architecture at Hot Chips next week.

The company has been embroiled in rumors of its Blackwell AI GPUs having issues big enough to require a redesign, and issues with Blackwell AI servers leaking through their water-cooling setups. NVIDIA has now shown its new Blackwell AI GPUs running in real-time, with Blackwell on-track to ramp into production and ship (in small quantities) to customers in Q4 2024.

NVIDIA also teased new pictures of various trays available in the Blackwell family, with these first images of Blackwell trays being teased showing just how much engineering and design work goes into these things. It's truly incredible, almost like a work of art... and that's on the outside.

NVIDIA CEO Jensen Huang will be flying over to Taiwan to attend SEMICON Taiwan 2024, where he will meet with SK hynix and TSMC executives according to local media.

Jensen will reportedly meet with SK hynix's president at SEMICON, where the two will provide the world with an update on their progress, as well as working with TSMC on next-generation HBM memory chips like HBM4.

NVIDIA, TSMC and SK hynix recently formed a "triangular alliance" for a collaborative effort to make next-gen AI GPUs and next-gen HBM memory. SEMICON Taiwan 2024 is a big event for the semiconductor industry, and with the triangular alliance there, we should expect some updates to GPU and HBM roadmaps, and more.

South Korean telecommunications giant SK Telecom has announced its partnering with US-based GPU cloud startup Lambda on a new GPU farm using NVIDIA H100 AI GPUs.

The GPU farm filled with NVIDIA H100 AI GPUs will be owned by Lambda, and installed into the SK Broadband data center in Gasan-dong, Guro-gu, Seoul, by December this year. The new partnership on the GPU farm in South Korea is part of SKT's broader plan to expand its AI data center business reports Business Korea.

SKT plans to scale the number of AI GPUs to thousands in the next 3 years, with SK Broadband being a subsidiary of SKT, and will play a "crucial role" in this venture by providing optimized equipment management services to ensure the stable operation of the GPU servers.

NVIDIA's upcoming earnings report is ruffling some feathers already, one analyst saying Blackwell AI GPUs design flaws are seeing the B100 AI GPU "effectively canceled".

The design flaws plaguing NVIDIA's new Blackwell AI GPUs hit headlines a couple of weeks ago, where we began hearing about design flaws that analyst firm KeyBanc says NVIDIA will need to "respin" the Blackwell tile that will cause a 3-month delay on shipments.

KeyBanc explained: "Given the Blackwell delay, we believe NVIDIA will prioritize the ramp of B200 for hyperscalers and has effectively canceled B100, which will be replaced with a lower cost/performance GPU (B200A) targeted at enterprise customers".

SK hynix Vice President Ryu Seong-su has announced the South Korean AI memory leader is working on developing a product with 20-30x the performance of current-gen HBM memory.

During the recent "SK Icheon Forum 2024" event held at the Grand Walkerhill in Gwangjin-gu, Seoul, South Korea, the SK hynix VP said: "We aim to develop products with 20 to 30 times the performance of current HBM, focusing on introducing differentiated products".

Ryu emphasized SK hynix's focus on responding to the mass market, with AI memory solutions (HBM) through execution capabilities, reports Business Korea. This strategy by SK hynix has been crucial in the demand of high-performance memory to grow, driven by the insatiable demand for AI GPU hardware.

Samsung is rumored to tape-out its next-generation HBM4 memory in Q4 2024, with mass production of HBM4 expected by the end of 2025.

The AI industry is fueling the growth of HBM memory, with HBM makers SK hynix, Samsung, and Micron pumping out as much as it can, and it hasn't been enough. SK hynix is sold out of all of its HBM3 and HBM3E for both 2024 and 2025, and with HBM4 on the horizon, all things are leading to who can be the biggest HBM4 supplier to AI GPU companies like NVIDIA and AMD.

NVIDIA's next-generation Rubin R100 AI GPU uses HBM4 memory, so SK hynix and Samsung are both doing everything they can (spending many tens of billions of dollars, setting up and expanding semiconductor fabs in South Korea, Taiwan, and the United States) to pump more HBM memory (HBM3, HBM3E, HBM4, and even HBM4E) for 2025 and beyond.

AMD has announced it is acquiring server maker ZT Systems for $4.9 billion, in a bid to better compete in the AI industry against juggernaut NVIDIA.

The company plans to pay for 75% of the ZT Systems acquisition with cash, while the remainder will be provided in AMD stock. As of Q2 2024, AMD has $5.34 billion in cash and short-term investments at the ready, and ZT Systems is one of those investments.

ZT Systems is an artificial intelligence infrastructure group that will help AMD create a better AI infrastructure, helping get AMD's new Instinct MI300 series AI accelerators into data centers. ZT Systems was founded over 30 years ago, and is a private company that builds custom computing infrastructure for some of the world's biggest AI hyperscalers.

You probably haven't heard of Ola, but they're an Indian automotive manufacturer, which has just announced they're going to design, build, and launch India's first-ever in-house AI chip by 2026 powered by the Arm architecture.

India is now jumping into the AI semiconductor market, with its first-ever in-house AI chip aimed at self-driving vehicles of the future. AI inside of autonomous vehicles is a big business now, and will be even bigger in the future and India wants in on that market.

Ola CEO Bravish Aggrawal took the stage at an event recently, announcing multiple AI chips are coming from Ola, each specializing in their respective scenarios. Aggarwal underlined the importance of India creating AI chips in-house, instead of relying on third-party semiconductor companies like TSMC, Samsung, or Intel. The company showed off some of its Bodhi series of AI chips, including the Sarv-1-cloud-native CPUs and its new Ojas edge AI chip.

Geekbench AI 1.0 is here, after years of feedback and test iteration with its customers, partners, and the AI engineering community, Primate Labs is proud to announce its latest machine learning benchmark is now ready, and it has a new name: Geekbench AI. You can download Geekbench AI 1.0 here.

Geekbench AI is a new benchmarking suite with a testing methodology for machine learning, deep learning, and AI-centric workloads, all with the same cross-platform utility and real-world workload reflection that Primate Labs' benchmarking software (Geekbench, duh) is known for.

The developer explains on its website: "Measuring performance is, put simply, really hard. That's not because it's hard to run an arbitrary test, but because it's hard to determine which tests are the most important for the performance you want to measure - especially across different platforms, and particularly when everyone is doing things in subtly different ways. At Primate Labs, we build our tests to reflect the sort of use cases that developers build their applications to do through detailed and ongoing conversations with software and hardware engineers across the industry, rather than just crunching basic math for hours".

SoftBank was talking with Intel about making an AI chip that would compete with NVIDIA's dominant AI GPUs, but the plan "floundered" after Intel failed to meet the requirements of Japan's SoftBank.

In a new report from the Financial Times, we're hearing that negotiations to partner with Intel would've rapidly accelerated SoftBank's efforts to combine the chip design of its "crown jewel" as the FT puts it: Arm. Japan's SoftBank owns Arm Holdings, so with its Arm architecture and Intel fabbing the chip with SoftBank's production expertise of its latest acquisition -- Graphcore -- according to "people familiar with the matter".

SoftBank boss Masayoshi Son plans to dump billions of dollars into putting Japan at the center of the AI boom, where the Arm owner wants to create a rival to NVIDIA's dominant AI GPUs. Son has pitched his ideas to the usual Big Tech companies, with chip production and software through to providing power for the data centers that would be powered by its chips.

Samsung's new 8-layer HBM3E memory has passed NVIDIA's qualification tests to be used in its AI GPUs.

In a new report from Reuters, the outlet says that the qualification clears a "major hurdle" for Samsung -- which is the world's largest memory manufacturer -- and has been struggling to catch up to South Korean rival, SK hynix, which has been providing its bleeding-edge HBM3E memory to NVIDIA for its AI GPUs.

Samsung still hasn't signed a supply deal for the approved 8-layer HBM3E memory chips with NVIDIA, but "will do so soon" according to Reuters' sources, who declined to be identified as the matter remains confidential. The new 8-layer HBM3E memory ships from Samsung would be supplied to NVIDIA in Q4 2024, which is not far away now.

The world of AI PCs is just beginning with the first waves of Copilot+ ready PCs out in the wild, with 14% of PCs shipped globally in Q2 2024 featuring an NPU, making them an AI PC.

In a new report from Canalys, we're learning that 8.8 million AI-capable PCs were shipped in Q2 2024, with each of those 8.8 million systems featuring the latest AI processor (with an NPU) from AMD, Apple, Intel, and Qualcomm. As we see new generations of AI processors from AMD and Intel, Canalys says that we can expect rapid growth in the second half of this year, and going into high gear in 2025.

If we look at just the Windows segment, AI-capable PC shipments surged 127% sequentially in Q2 2024, with Lenovo shipping out its Qualcomm Snapdragon X Elite-based Copilot+ PCs with the Yoga Slim 7x and ThinkPad T14s, boosting its AI PC market share to around 6% of all Windows PC shipments, a mammoth 228% growth. HP followed behind Lenovo with a 7% share of AI PCs.

It looks like there is a "component shortage" for NVIDIA's new Blackwell-based GB200 AI servers, with market demand in Taiwan at the highest levels because of the AI market needing as many chips as possible.

In a new report from Taiwan Economic Daily, we're learning that NVIDIA's new GB200 AI servers are experiencing huge development issues, with the main problem stemming from leaks from the liquid cooling system inside of the next-generation multi-million-dollar AI server.

The water leakage inside of a GB200 AI server would be a disaster as you can imagine, with Taiwanese manufacturers moving into emergency mode to solve it, with new Taiwanese suppliers stepping up to help NVIDIA get its GB200 AI servers to market.

SK hynix has just announced it's planning to develop a 4F2 (square) DRAM, joining South Korean rival Samsung and its journey into the world of 3D DRAM.

The cost of EUV (extreme lithography) processes has continued to skyrocket since the commercialization of 1c DRAM, with SK hynix researcher Seo Jae Wook noted during an industry conference in Seoul, South Korea, on Monday.

The Elec reports that Seo said at the time whether manufacturing DRAM this way (using EUV) was profitable, where in response, SK hynix said it was considering manufacturing vertical gate (VG) or 3D DRAM for future DRAM. VG is what SK hynix internally calls 4F2, while Samsung calls theirs vertical channel transistor (VCT).

Last month we reported on AMD's announcement that it was acquiring the largest private AI lab in Europe - Silo AI. Today, AMD has followed up on the announcement to confirm that the "all-cash transaction valued at approximately $665 million" has now been completed, with Silo AI's scientists and engineers now a part of the AMD family.

$665 million in cash is nothing to sneeze at, and for AMD, the acquisition is the latest step in the company's broader pivot that puts its main focus on AI and AI-related technologies. This is nothing new; we've seen the same shift happen in other companies like Google, Meta, Apple, and, of course, NVIDIA. However, NVIDIA's AI focus started many years ago.

"AI is our number one strategic priority," said Vamsi Boppana, AMD senior vice president, AIG. "We continue to invest in both the talent and software capabilities to support our growing customer deployments and roadmaps."

GPU VRAM modding isn't new, but Chinese cloud companies are adding copious amounts of VRAM to the likes of the GeForce RTX 4090D (24GB modded to 48GB) and the RTX 4080 SUPER from 16GB to 32GB and renting them out at an hourly rate.

Now, we've got a Chinese AI expert that is talking with people in the country about selling NVIDIA's new GeForce RTX 4090D graphics card -- but with double the VRAM, from 24GB to 48GB of GDDR6X -- as well as the GeForce RTX 4080 SUPER upgraded from 16GB to 32GB of GDDR6X memory.

The source says that the GPUs are popular with cloud computing companies, as they're renting out the spare AI computing power because the demand is so high right now. The modded GeForce RTX 4080 SUPER with 32GB of VRAM is available for just $0.03 per hour, which is a damn good price for a monster modded RTX 4080 SUPER with 32GB of GDDR6X memory for AI workloads.

South Korea's memory chip exports to Taiwan surged over 225% year-over-year, thanks to the unstoppable demand of AI GPUs and AI accelerators and their respective HBM memory chips.

In a new report from the Korea Times, we're learning that outbound shipments of HBM memory chips to Taiwan reached $4.26 billion in the first 6 months of this year, up an incredible 225.7% from a year ago, "far outperforming" South Korea's overall increase of memory chip exports at 88.7%, according to data compiled by the industry ministry and the Korea International Trade Association.

Taiwan was the third-largest importer of South Korean memory chips in the period, pushing out both Vietnam and the United States. Market watchers said that the huge surge in memory chip exports to Taiwan is thanks to SK hynix's fantastic HBM supply to NVIDIA, which uses its HBM memory inside of its AI GPUs, and fabs its chips over at TSMC in Taiwan.

NEO Semiconductor has just unveiled the development of its new 3D X-AI chip technology, which aims to replace DRAM chips inside of HBM to solve data bus bottlenecks, by enabling AI processing through 3D DRAM.

The new 3D X-AI chip technology can reduce the huge amount of data transferred between HBM and GPUs during AI workloads, with NEO's innovative new 3D X-AI chip technology set to "revolutionize the performance, power consumption, and cost of AI chips for AI applications like generative AI".

NEO's new 3D X-AI chip technology has 100x the performance with 8000 neuron circuits to perform AI processing in 3D memory, a huge 99% power reduction that minimizes the requirement of transferring data to the GPU for calculation, reducing power consumption and heat generation by the data bus, and 8x the memory density with 300 memory layers, allowing HBM to store larger AI models.

When it comes to AI training and dealing with large data sets and increasingly complex large language models (LLMs) like Llama 70b, it's not simply a matter of being able to throw GPU horsepower at the problem until you find a solution. At least, it shouldn't be.

Phison's aiDAPTIV+ is a hybrid software and hardware solution for LLM training. It integrates Phison's Pascari A100 M.2 SSDs into a complete solution with linear scaling. Impressive! According to Phison, it unlocks access to run workloads previously reserved for data centers on a single workstation or server - supporting up to Llama-3 70B and Falcon 180B.

Phison's chart showcases the capabilities of aiDAPTIV+. Above, you can see a single system with the same configuration: four RTX 6000 Ada GPUs, 192GB of GDDR6 memory, and an additional 512GB of RAM. Phison's aiDAPTIVLink middleware extends this GPU memory capacity with two 2TB SSDs, paving the way for massive model support with low latency. This is impressive stuff, and it won the "Best of Show, Most Innovative AI Application" award at FMS: the Future of Memory and Storage.

It was only last month we heard about Apple, NVIDIA and many other big name players in the AI race being caught up in an investigative report that found they all used a public data set containing YouTube video transcripts to train their respective AI products, which is a violation of YouTube's terms-of-service (TOS).

YouTube has said in the past that any "unauthorized scraping or downloading of YouTube content" is strictly prohibited, and it's especially prohibited when that data is then used for commercial projects. Last month, a Proof News investigation found NVIDIA, Apple, and other AI companies used an academic data set containing subtitles from more than 170,000 YouTube videos to train AI models, and now NVIDIA has been caught in the spotlight again with a report from 404 Media.

According to the publication that spoke with a former NVIDIA employee about the company's internal processes, employees were instructed to scrape videos from Netflix, YouTube, and other sources to add to the data sets that are being used to an AI model for NVIDIA's Omniverse 3D world generator, self-driving car systems, a "digital human" AI avatar product, and the Cosmos deep learning model.