Artificial Intelligence - Page 51

NVIDIA spent a sizeable $10 billion on R&D for its next-generation Blackwell GPU architecture, but chip legend Jim Keller said he could've done the same job for just $1 billion.

I first saw Keller's tweet, noticing that it looked familiar... he had taken an image of the story I wrote about NVIDIA spending $10 billion on R&D for Blackwell. Great to see, but how does Keller fix NVIDIA's $10 billion R&D budget for one-tenth of that cost, just $1 billion?

First, NVIDIA uses dual Blackwell B100 dies on its new B200 AI GPU, which has a whopping 208 billion transistors, each B100 GPU die featuring 104 billion transistors. NVIDIA uses two B100 dies to create the B200 in full, with NVIDIA using its in-house NV-High Bandwidth Interface (NV_HBI), which has up to 10TB/sec of bandwidth.

LG has announced plans to expand its home appliance-specific on-device AI chip -- DQ-C -- to 46 models through 8 product families.

The new LG DQ-C chip supports AI control, LCD display driving, and voice recognition and is specialized for operating systems inside of home appliances. LG has designed the DQ-C AI chip in-house, with previous chips produced by other semiconductor companies, while LG outsources to TSMC on its 28nm process node in Taiwan to make its DQ-C AI chip.

LG has spent three years deep inside research and development of the DQ-C chip, first announced in July 2023, and is used inside of five LG products including washing machines, dryers, and air conditioners. LG first introduced washing machines and dryers with its DQ-C chip under its Home Appliances 2.0 series in July 2023, showing off the actual DC-Q chip at IFA last year.

The latest IEA Electricity 2024 report states that the electricity and power demands from the data center sector in countries like the United States and China will increase dramatically by the time 2030 rolls around. If you've been keeping track of some of the AI data center plans from the likes of Google, Meta, Amazon, and others, you're aware that this year alone, hundreds of thousands of high-performance NVIDIA GPUs are set to be installed in various locations.

The report writes, "The AI server market is currently dominated by tech firm NVIDIA, with an estimated 95% market share. In 2023, NVIDIA shipped 100,000 units that consume an average of 7.3 TWh of electricity annually. By 2026, the AI industry is expected to have grown exponentially to consume at least ten times its demand in 2023."

According to Rene Has, CEO of Arm (via The Wall Street Journal), if AI accounts for around 4% of current power usage in the United States, this could rise to around 25% by 2030. He also called out generative AI models like Chat GPT as "insatiable" regarding electricity.

NVIDIA's shortage of Hopper H100 AI GPUs is improving, with the previous 4-month wait now turning into 8-12 weeks.

It was just a few months ago that we reported that NVIDIA AI GPU shipments had been "greatly accelerated," according to analysts, with waiting times of 8-11 months for AI GPU deliveries reduced to just 3-4 months. Now that 4-month wait, is a 2-3 month wait.

In a new report from TrendForce, Dell is reportedly capitalizing on AI, with Dell Taiwan's General Manager saying on April 9 that the company is experiencing stronger server orders and demand in the Taiwanese market. This surge is thanks to AI needs within Taiwan's own corporate sector.

The next wave of powerful AI-powered chatbots is only just around the corner as Meta and OpenAI prepare for the release of GPT-5 and Llama 3, the large language models that power popular AI tools such as ChatGPT.

The underpinning technology powering popular AI tools such as ChatGPT and DALL-E will soon be getting an upgrade, according to recent reports citing progress updates from Meta and OpenAI, the two tech giants leading the charge when it comes to AI development.

Meta's president of global affairs Nick Clegg said the company is currently preparing to release Llama 3 to the public "Within the next month, actually less" and that this next-generation of Llama will arrive with a suite of new features that Meta promises will be much more impressive than the current model.

With all the big tech companies investing billions in AI data centers, research, and the creation of generative AI models and tools, many are looking to create their own hardware as an alternative to NVIDIA's chips - while competing with AMD, Intel, and new AI-chip players like Microsoft.

Google is entering the race with its own arm-based processor designed for the AI market. Like Google's tensor processing units (TPUs), which developers can access only via Google Cloud, the Arm-based CPU called Axiom will apparently deliver "superior performance to x86 chips."

How much extra performance? According to Google, Axiom offers 30% better performance than "general purpose Arm chips" and 50% better performance than "current generation x86 chips" as produced by Intel and AMD.

Meta has just teased its next-gen AI chip -- MTIA -- which is an upgrade over its current MTIA v1 chip. The new MTIA chip is made on TSMC's newer 5nm process node, with the original MTIA chip made on 7nm.

The new Meta Training and Inference Accelerator (MTIA) chip is "fundamentally focused on providing the right balance of compute, memory bandwidth, and memory capacity" that will be used for the unique requirements of Meta. We've seen the best AI GPUs on the planet using HBM memory -- with HBM3 used on NVIDIA's Hopper H100 and AMD Instinct MI300 series AI chips -- with Meta using low-power DRAM memory (LPDDR5) instead of server DRAM or LPDDR5 memory.

The social networking giant created its MTIA chip was the company's first-generation AI inference accelerator that the company designed in-house for Meta's AI workload in mind. The company says that their deep learning recommendation models are "improving a variety of experiences across our products".

AMD has already confirmed it has refreshed variants of its Instinct MI300 series AI and HPC processors in the second half of this year, with a tweaked Instinct MI350X featuring ultra-fast HBM3E memory.

AI GPU competitor NVIDIA has its current Hopper H100 AI GPU with HBM3 memory, while its newly announced H200 AI GPU features ultra-fast HBM3E memory -- the world's first AI GPU with HBM3E memory. The next-gen Blackwell B200 AI GPU ships with ultra-fast HBM3E memory as standard.

Market research firm TrendForce recently teased AMD's new Instinct MI350X. The firm says the new Instinct MI350X will feature chiplets made on TSMC's newer 4nm process node, which is an enhanced version of TSMC's 5nm-class process node. The new TSMC N4 process node will allow AMD to choose between increasing performance or lowering power consumption on its tweaked Instinct MI350X over the MI300 series AI GPU.

Intel has recently held its Vision Keynote where the company's CEO Pat Gelsinger touched on the current state of the technology industry and how AI will be implemented into businesses and companies around the world.

Gelsinger took to the stage and began his discussion by describing this as the age of AI, and how he believes that in the not-so-distant future AI tools will begin interacting with other AI tools to complete tasks, which will result in entire departments becoming automated by AI bots. Gelsinger added that expanding this idea of having AI-automated departments may even result in the very first one-person, billion-dollar company, which is referred to as a "Unicorn".

As you can probably imagine, Gelsinger touted the power of Intel powering the mass adoption of AI throughout businesses and even at home, even going as far to say that he calls this the age where every company becomes an AI company, which will drive the semiconductor TAM [total addressable market] from approximately $600 billion to more than $1 trillion by the end of the decade.

US Congressman Adam Schiff is attempting to force AI companies to outline any copyrighted data used to train AI models.

On April 9, Schiff introduced the Generative AI Copyright Disclosure Act that, if passed, would require AI companies to file all relevant data used to train their AI tools with the Register of Copyrights at least 30 days before the tool is introduced to the public.

The bill would also be retroactive, meaning any AI tool that is currently available to the public would fall under the same new requirement. If the company doesn't comply with the new laws, it will face a financial penalty from the Copyright Office proportionate to the company's size and violations.

It was only a few days ago that YouTube's CEO put out a warning directed at OpenAI reminding the company that using any data acquired from its video platform will be a violation of its terms of use.

Now, reports are surfacing from The New York Times that OpenAI has trained its most advanced AI model, GPT-4, with more than a million hours of transcribed YouTube videos, according to sources that spoke to the newspaper and told it audio and video transcripts were fed into the company's latest AI model. Moreover, these sources also said that Google, the owner of YouTube, has also used audio and video transcripts to train its AI models, both of which are clear violations of YouTube's terms of use.

A spokesperson for Google, Matt Bryant, told the NYT that any "unauthorized scraping or downloading of YouTube content" is prohibited. It should be noted that the NYT has filed a lawsuit against OpenAI and Microsoft for copyright infringement, alleging the company took the newspaper's content without permission.

Intel has just unveiled its next-gen Gaudi 3 AI accelerator, which features two 5nm dies made by TSMC, featuring 64 Tensor Cores (5th Generation), 128GB of HBM2e memory, and up to 900W of power on air or water-cooling.

Intel has 32 Tensor Cores on each chip, for a total of 64 Tensor Cores. Each chip features 48MB of SRAM, for a total of 96MB of SRAM per full package. The SRAM on the Intel Gaudi 3 AI accelerator features 12.8TB/sec of bandwidth, supported by the HBM memory on the Gaudi 3. The 128GB of HBM2e memory features up to 3.7TB/sec of memory bandwidth.

The previous-gen Intel Gaudi 2 AI accelerator featured 96GB of HBM, so the new Gaudi 3 has a bigger 128GB HBM2e capacity with up to 3.7TB/sec of memory bandwidth compared to just 2.45TB/sec of memory bandwidth on Gaudi 2.

Elon Musk has predicted that the development of artificial intelligence will get to the stage of being smarter than the smartest humans by 2025, and if not, by 2026.

In an explosive interview on X Spaces, the Tesla and SpaceX boss told Norway wealth fund CEO Nicolai Tangen that IA was constrained by electricity supply and that the next-gen version of Grok, the AI chatbot from Musk's xAI startup, was expected to finish training by May, next month.

When discussing the timeline of developing AGI, or artificial general intelligence, Musk said: "If you define AGI (artificial general intelligence) as smarter than the smartest human, I think it's probably next year, within two years". A monumental amount of AI GPU power will be pumped into training Musk's next-gen Grok 3, with 100,000 x NVIDIA H100 AI GPUs required for training.

Elon Musk has talked about training his next-gen Grok 3 AI chatbot, saying it will require an insane 100,000 NVIDIA H100 AI GPUs.

During a recent X spaces chat, the SpaceX and Tesla boss said that Grok 2 used around 20,000 NVIDIA H100 AI GPUs to train, but the new Grok 3 training will require a monster 100,000 separate NVIDIA H100 AI GPUs, a mammoth amount of AI compute power.

Musk said that the upcoming Grok model and beyond will require 100,000 NVIDIA H100 AI GPUs, so we can expect Grok 4 to require an unimaginable amount of AI GPU compute power. NVIDIA has now announced its new Blackwell B200 AI GPU, which I'm sure Elon has been eyeing off... which will be pumped out later this year, and flood the market in 2025 with brute force AI GPU performance.

Samsung has announced that it has manufactured a sample of its brand-new 16-stack HBM. The company manufactured the chip with hybrid bonding last week.

Samsung vice president Kim Dae-woo said the South Korean giant made its new 16-layer HBM with HBM3 but plans to use HBM4 to improve productivity. Due to alignment issues, Samsung was expected to use hybrid bonding for just one or two stacks of the HBM chip but applied the hybrid bonding technique to all 16 stacks.

The 16-stack HBM memory sample was made using equipment from Samsung's fab equipment subsidiary, Semes, with Kim noting that Samsung considered using hybrid bonding or thermal compression non-conductive film for HBM4, which Samsung will be sampling in 2025 and mass producing next-gen HBM4 memory in 2026.

We know SpaceX and Tesla boss Elon Musk loves hardware as much as he loves AI, so it's no surprise that he's posting on X that Tesla has the second-highest H100 AI GPU count in the world.

In a reply to @thetechbrother on X who posted that Meta has 350,000+ NVIDIA H100 AI GPUs, with a bunch of other companies -- including Tesla -- and how many H100 AI GPUs they've got so far. Elon replied to that, saying "this is not accurate. Tesla would be second highest and X/xAI would be third if measured correctly".

Meta has 350,000+ NVIDIA H100 AI GPUs right now, so if Tesla had the second highest, the electric vehicle giant would have somewhere between 30,000 and 350,000 H100 AI GPUs. Lambda in the US is the second-highest on these charts, with 30,000 H100 AI GPUs in operation, but now we know from the horses mouth himself -- Elon Musk -- that Tesla is second there, with somewhere between 30K and 350K H100 AI GPUs. That's a lot of AI compute power.

Samsung has reportedly won a contract with NVIDIA to provide the AI GPU giant with advanced 2.5D packaging.

The news is coming from TheElec, with their sources saying Samsung's Advanced Package (AVP) team will be providing an interposer and I-Cube -- its 2.5D package -- to NVIDIA. Other companies will produce the High-Bandwidth Memory (HBM) and GPU wafers, with the 2.5D packaging housing the chip dies-CPU, GPU, I/O, HBM, and others-placed horizontally onto the interposer.

Samsung calls its 2.5D packaging technology I-Cube, while TSMC calls its 2.5D packaging CoWoS (Chip-on-Wafer-on-Substrate). NVIDIA's entire fleet of A100 and H200 series AI GPUs uses 2.5D packaging, and more importantly, the monster new 208 billion transistor Blackwell B200 AI GPU uses the same advanced packaging.

Jony Ive, Apple's former head of design, is reportedly working with OpenAI CEO Sam Altman on a new AI-powered personal device with the pair now seeking funding for the new project.

The news, shared by The Information, means that the pair have teamed up on what could be a new device similar to the Humane AI pin or something along those lines. Notably, Altman is also a major investor in Humane so there are clear links there.

Details on exactly what the product will be are hard to come by right now but it apparently won't be like a smartphone, something that will surely be music to the ears of Apple's executives. This also isn't the first time that we've heard that the pair are working together after information surfaced last fall. However, things seem to have now progressed somewhat with the two people now thought to be seeking funding to the tune of $1 billion.

Could AI spell doom for humankind eventually? Depending on which expert you talk to, the chances vary considerably, but one researcher definitely has a gloomy (and doomy) opinion - one that Elon Musk doesn't share.

Business Insider reported on revelations made at the recent Abundance Summit (held last month), which included a 'great AI debate' where Musk estimated the risk of AI ending humanity was "about 10% or 20% or something like that."

Obviously that's something akin to wild guesswork, but the general gist of the billionaire's philosophy is that we should push ahead with AI development as the probable positive outcomes outweigh any negative scenario.

Apple is diving head first into artificial intelligence-powered systems and according to reports citing Apple researchers behind these new systems, one specifically is designed to take on OpenAI's GPT products.

Reports indicate that Apple is developing the ReaLM system, which stands for "Reference Resolution As Language Modeling", a new system that is designed to make interacting with AI much more natural. Additionally, ReaLM is able to "see" on-screen content, with the researchers behind the project saying it outperforms OpenAI's GPT-4, the underlying technology powering ChatGPT, in determining context and interpreting linguistic expressions.

Additionally, the researchers behind the project believe ReaLM is "an ideal choice" for a context deciphering system that would be present "on-device without compromising on performance". So, how would it work? Imagine asking Siri to show you a list of local groceries around your location. After Siri has completed bringing that list up you will then be able to ask her "Call the bottom one." With ReaLM implementation Siri would be able to identify the bottom option and proceed to call them. Apple researchers say ReaLM outperformed GPT-4 in this context deciphering area.