Artificial Intelligence - Page 36

NVIDIA might have its new Blackwell AI GPU architecture slowly coming out, but its Hopper H100 and new H200 AI GPUs are continuing to get even stronger with new optimizations in the CUDA stack.

The H200 and H100 AI GPUs offer leading performance across every single text compared to the competition, including the latest benchmarks like the 56 billion parameter "Mixtral 8x7B" LLM.

NVIDIA's monster HGX H200 packing 8 x Hopper H200 GPUs and NVSwitch has some strong performance gains in Llama 2 70B, with a token generation speed of 34,864 (offline) and 32,790 (server) with a 1000W and 31,303 (offline) and 30,128 (server) in the 700W config.

OpenAI is rumored to have its first in-house AI chip made by TSMC on its new angstrom process (A16) production lines, according to the latest rumors.

UDN is reporting that OpenAI's first self-developed AI chip will be made by TSMC on its A16 process node, with its new AI chips said to power OpenAI's text-to-video service: Sora. OpenAI's new Sora text-to-video service will be a "major selling point" of Apple's AI in the future, reports UDN.

The big boost of OpenAI's new in-house AI chip will "boost" Apple's efforts to attack the AI market, and to stimulate the development of related networking and high-speed transmission industries. The legal person who spoke with UDN said that Sora has stimulated a surge in demand for data transmission speeds, and that silicon photonics and high-speed optical modules can increase those data transmission speeds.

Microsoft is finally ready to enter the custom AI hardware race, a chip market in which NVIDIA has at least a 75 percent market share. At this year's Hot Chips conference, the company unveiled its first AI accelerator, Maia 100, built on TSMC's 5nm process node.

Designed to "optimize performance and reduce costs," Maia 100's architecture includes custom server boards, racks, and software for running AI services like Microsoft's Azure OpenAI Services.

"The Maia 100 accelerator is purpose-built for a wide range of cloud-based AI workloads," Microsoft's technical blog on Maia 100 details. "The chip measures out at ~820mm2 and utilizes TSMC's N5 process with COWOS-S interposer technology. Equipped with large on-die SRAM, Maia 100's reticle-size SoC die, combined with four HBM2E die, provide a total of 1.8 terabytes per second of bandwidth and 64 gigabytes of capacity to accommodate AI-scale data handling requirements."

OpenAI is one of biggest companies in the world leading the push into artificial intelligence-powered surfaces, and you could certainly argue the company was the first to popularize the technology that has been used by developers in less sophisticated forms for quite some time.

Artificial intelligence-powered devices, services, and applications are popping up everywhere, and the moniker AI labeling is being slapped on seemingly every piece of technology that it can be applied to - even some that don't deserve it and are simply attempting to ride the hype surrounding AI products.

The popularity of AI can be traced back to the explosion that was the release of OpenAI's ChatGPT, in which it attracted more than 100 million monthly active users in just two months, setting the record for the fastest growing consumer-application in history. ChatGPT was released in November 2022, and according to OpenAI it's active monthly user base has grown substantially since then, attracting an additional 100 million users.

NVIDIA is thinking about joining the new funding round for OpenAI, that would value the AI startup at over $100 billion, reports The Wall Street Journal.

According to "people familiar with the matter" the new valuation would peg OpenAI at worth over $100 billion, and would see NVIDIA joining US-based tech giants Apple and Microsoft who are also reportedly in talks to participate in the funding round.

Thrive Capital would be leading the funding round, investing around $1 billion, while two sources of the WSJ said that NVIDIA discussed investing about $100 million into OpenAI. The Wall Street Journal reached out to representatives from NVIDIA, Apple, Microsoft, OpenAI, and Thrive, but all of them declined to comment (no surprise).

OpenAI has announced its ChatGPT chatbot service has over 200 million weekly active users, double the 100 million users that were using ChatGPT this time last year. That's a big upgrade in users, but ChatGPT has taken over the world.

ChatGPT was launched in 2022 capable of generating human-like responses based on user prompts, and had over 100 million weekly active users, OpenAI CEO Sam Altman said in November. The AI startup said that 92% of Fortune 500 companies are using its AI-powered products, and the use of its automated Application Programming Interface (API) allowing software programs to talk to each other, has double since ChatGPT-4o mini launched in July 2024.

ChatGPT-4o mini is a cost-effective, smaller AI model that is aimed at making its AI-powered technology more affordable, and it uses less power, allowing OpenAI to target a wider range of customers.

Just before NVIDIA announced its issues with its new Blackwell AI GPUs, partner Supermicro seemingly confirmed Blackwell B200 AI GPUs being delayed, offering its customers liquid-cooled Hopper H200 AI GPUs in their place.

Supermicro CEO Charles Liang said that the possible delay of NVIDIA's new Blackwell GPUs for AI and HPC systems will not have a dramatic impact on AI server makers, or the AI server market. Liang said: "We heard NVIDIA may have some delay, and we treat that as a normal possibility".

Liang continued: "When they introduce a new technology, new product, [there is always a chance] there will be a push out a little bit. In this case, it pushed out a little bit. But to us, I believe we have no problem to provide the customer with a new solution like H200 liquid cooling. We have a lot of customers like that. So, although we hope better deploy in the schedule, that's good for a technology company, but this push out overall impact to us. It should be not too much".

Yep, NVIDIA has admitted it has had issues with its new Blackwell AI GPUs that are causing low yields, forcing the company to re-spin some of the layers of its new B200 AI GPU to boost yields.

NVIDIA said in a statement: "We executed a change to the Blackwell GPU mask to improve production yield. Blackwell production ramp is scheduled to begin in the fourth quarter and continue into fiscal 2026. In the fourth quarter, we expect to ship several billion dollars in Blackwell revenue".

The design flaws plaguing NVIDIA's new Blackwell AI GPUs hit headlines a couple of weeks ago, where we began hearing about design flaws that analyst firm KeyBanc says NVIDIA will need to "respin" the Blackwell tile that will cause a 3-month delay on shipments. Now these reports ring true. KeyBanc explained at the time: "Given the Blackwell delay, we believe NVIDIA will prioritize the ramp of B200 for hyperscalers and has effectively canceled B100, which will be replaced with a lower cost/performance GPU (B200A) targeted at enterprise customers".

Google's research scientists have published a paper on its new GameNGen technology, an AI game engine that generates each new frame in real-time based on player input. It kind of sounds like Frame Generation gone mad in that everything is generated by AI, including visual effects, enemy movement, and more.

AI generating an entire game in real-time is impressive, even more so when GameNGen uses its tech to recreate a playable version of id Software's iconic Doom. This makes sense when you realize that getting Doom to run on lo-fi devices, high-tech gadgets, and even organic material is a right of passage.

Seeing it in action, you can see some of the issues when it comes to AI generating everything (random artifacts, weird animation), but it's important to remember that everything you see is being generated and built around you in real-time as you move, strafe, and fire shotgun blasts at demons.

AMD's new Instinct MI300X AI accelerator with 192GB of HBM3E has had a deep dive at Hot Chips 2024 this week, as well as the company teasing its refreshed MI325X with 288GB of HBM3E later this year.

Inside, AMD's new Instinct MI300X AI Accelerator features a total of 153 billion transistors, using a mix of TSMC's new 5nm and 6nm FinFET process nodes. There are 8 chiplets that feature 4 shared engines, and each shared engine contains 10 compute units.

The entire chip packs 32 shader engines, with a total of 40 shader engines inside of a single XCD and 320 in total across the entire package. Each individual XCD has its dedicated L2 cache, and out the outskirts of the package, features the Infinity Fabric Link, 8 HBM3 IO sites, and a single PCIe Gen5 link with 128GB/sec of bandwidth that connects the MI300X to an AMD EPYC CPU.