Artificial Intelligence - Page 43

Intel announced its new Gaudi 3 AI accelerator earlier this year, rocking up to 128GB of HBM2e memory with up to 3.7TB/sec of memory bandwidth... but how does it stack up against NVIDIA's dominant Hopper H100 80GB AI GPU?

Well, that comes down to cost -- Intel's new Gaudi 3 AI accelerator costs just $16,000 per AI accelerator with 128GB of HBM2e memory, compared to $30,000+ for NVIDIA's Hopper H100 AI GPU with 80GB of faster HBM3 memory. Intel is charging around $65,000 per 8 x Gaudi 2 accelerators on a baseboard, and $128,000 per 8 x Gaudi 3 AI accelerators on a baseboard.

This means that a single Gaudi 2 AI accelerator costs around $8125, while Gaudi 3 costs $15,650, when purchased in bulk, with the baseboards, of course.

NVIDIA's new GB200 Superchip is getting another supplier according to analyst Ming-Chi Kuo, reporting that Alpha and Omega Semiconductor Ltd (AOSL) is to join the GB200 AI server train.

In a new post on Kuo's own Medium blog and a post on X, the analyst said: "Alpha and Omega Semiconductor Ltd (AOSL) is poised to become a new GB200 supplier and one of the leading beneficiaries of NVIDIA's certification process change".

NVIDIA CEO Jensen Huang said recently that the company will be releasing new versions of its flagship AI chips annually, and while this will absolutely enhance NVIDIA's competitiveness, it introduces other issues. Execution, acceleration of supplier certification processes, as well as accelerating system development.

TSMC and contract chip design partner Global Unichip have won the bulk orders for base dies used with SK hynix's next-generation HBM4 memory chips.

In a new report from UDN, we're learning that after being the exclusive OEM for AI chips for technology giants like NVIDIA and AMD, TSMC is seizing another AI business opportunity. UDN reports that TSMC will collaborate with its special application IC (ASIC) design service factory Creative, now successfully developing the HBM key peripheral components required for AI servers, jointly inking a "large order" for base dies required for next-gen HBM4 memory.

UDN's report says that TSMC and Creative have "never commented on order developments," but the legal source of the site said that demand for AI is strong, and not just HPC-related chips, but HBM demand is "growing rapidly" and is "becoming a new business opportunity in the market" which has three major memory chip companies including SK hynix, Samsung, and Micron, investing as much as they can into their respective HBM supply chains.

NVIDIA's new Blackwell GB200 and B-series AI GPUs, including the B100 and B200, have received a "large number of customers," and that "demand exceeds supply," reports UDN.

After NVIDIA aggressively increased the production volume of TSMC's advanced process, the "order-chasing effect" spread to the back-end packaging and testing plants. ASE Investment Holdings and KYEC operations have "exploded," reports UDN, with related order volume doubling in the fourth quarter.

UDN reports from its sources that KYEC is "full of new orders" from NVIDIA, that it is moving internally to welcome those large volumes of orders, and that it needs to be relocated for this purpose alone. More production capacity can then meet NVIDIA's growing (unstoppable) needs.

Because of US sanctions, NVIDIA has been restricted from supplying multiple countries across the Middle East, but more importantly, China has been restricted from shipping its leading AI GPUs.

It appears that might be changing with a new report from Reuters teasing that NVIDIA believes it can get an export license to sell its AI GPUs to a telecom giant that operates in multiple Middle East countries. NVIDIA has reportedly inked a deal with Ooredoo to "introduce its AI technology" in datacenters across five Middle Eastern countries.

This would mark NVIDIA's first major expansion that has US export regulations stamped on it, somewhere that the company isn't allowed to provide its AI GPUs to: the Middle East (it's not just China). Ooredoo is a telecom giant that operates in the Middle East, and will soon be powered by NVIDIA AI and HPC GPUs across its datacenters located in Qatar, Algeria, Tunisia, Oman, Kuwait, and the Maldives.

Apple has announced it will be pausing the rollout of its Apple Intelligence across Europe as the company has to adhere to the European Union competition regulations.

Apple Intelligence was announced at WWDC a few weeks ago and is expected to be released alongside iOS 18 and the iPhone 16. Moreover, Apple Intelligence is Apple's foray into artificial intelligence, as the new update to Apple's operating system will integrate a slew of AI-powered features, and completely overhaul Siri.

However, Apple has said in a statement on Friday that specific iOS 18 features won't be rolled out in the EU, and those are: Phone Mirroring, SharePlay Screen Sharing enhancements, and Apple Intelligence.

HBM is the key to AI chips of today and more especially, the future, with HBM production capacity at its limits as AI chip makers like NVIDIA, have been scooping it all up.

SK hynix, Samsung, and Micron lead the HBM memory chip side, with HBM3 and HBM3E orders from all three major DRAM manufacturers completely out of stock until the end of 2025. We've been reporting on that side of the industry heavily, but now all three HBM manufacturers -- SK hynix, Samsung, and Micron -- are expanding HBM production capacity to keep up with the continuously growing demand of the AI market.

This means that the demand for application materials outsourcing equipment and components from the three leading DRAM factories has been "set aside to next year" reports UDN, with their legal person pointing out that due to multiple new customer orders from major semiconductor equipment manufacturers has pushed the order visibility into Q2 2025, meaning that performance of the markets is expected to be strong throughout this year, and into 2025 easily.

NVIDIA has been pushing new technologies into gaming through ray tracing, DLSS, AI-powered technologies, and so much more... but we all know that AI is the future of gaming.

NVIDIA CEO Jensen Huang said one of those AI-powered technologies could be similar to how RTX Remux acts, but would do it in real-time. Jensen was asked recently "AI has been used in games for a while now, I'm thinking DLSS and now ACE. Do you think it's possible to apply multimodality AIs to generate frames?"

Jensen replied: "AI for gaming - we already use it for neural graphics, and we can generate pixels based off of few input pixels. We also generate frames between frames - not interpolation, but generation. In the future we'll even generate textures and objects, and the objects can be of lower quality and we can make them look better. We'll also generate characters in the games - think of a group of six people, two may be real, and the others may be long-term use AIs".

Elon Musk's startup xAI has made a massive order with various hardware companies to create a powerful supercomputer that will be used to power Grok.

The CEO of Dell Technologies, Michael Dell, took to his personal X account to announce that Dell has partnered with xAI to construct a "Dell AI factory". Dell isn't the only company onboard with the supercomputer project, as NVIDIA will provide the horsepower with its powerful AI GPUs, and Supermicro will handle one-half of the total number of racks. Elon Musk clarified on his personal X account, "Dell is assembling half of the racks that are going into the supercomputer that xAI is building."

As for the horsepower, Musk stated earlier this month that xAI was on the verge of bringing 100,000 liquid-cooled NVIDIA H100 GPUs online, but Musk isn't totally thrilled with their efficiency, especially with NVIDIA's next-generation Blackwell GPUs on the horizon. Musk stated earlier this month, "Given the pace of technology improvement, it's not worth sinking one gigawatt of power into H100s. Next big step would probably be ~300k B200s with CX8 networking next summer."

SK hynix is expediting its HBM roadmap that includes HBM4 and HBM4E memory, which were originally planned for mass production in 2026 and 2027, respectively.

Now, the timelines have been moved up, and HBM4 is set for mass production in 2025, while HBM4E will enter mass production in 2026, according to sources from Business Korea. The adjustments "align" with NVIDIA's accelerated AI accelerator release cycle, which has shortened from two years to one year.

NVIDIA's current-gen Hopper GPU architecture with the H100 and beefed-up H200 are already dominating, with the new Blackwell GPU architecture with the B100 and B200 AI GPUs sporting faster HBM3E memory. But, then NVIDIA teased its next-gen Rubin R100 AI GPU, which will feature HBM4 memory and drop in Q4 2025.

NVIDIA still hasn't signed on with Samsung for its HBM memory chips, something that will need to be resolved by NVIDIA because it requires Samsung's silicon interposers and advanced chip packaging.

In a new report from TheElec, the huge bottleneck for NVIDIA's current-gen Hopper H100 and H200 AI GPUs is now a shortage of TSMC's CoWoS advanced packaging capacity, with an interposer shortage on the horizon. The interposer is another important key part of AI chips, with the interposer integrating multiple chips into a single package.

NVIDIA's advanced AI GPU itself, HBM memory chips, and more. H100's interposer is 3.3x the size of a lithography mask, and only 9 can be cut form a 12-inch silicon wafer (on the 40nm process). However, the more advanced GPUs like NVIDIA's new Blackwell B100 and B200 AI GPUs are even bigger, and by 2026, with Rubin R100 AI GPUs, interposers will need to be 5.5x the size of a mask... by 2027, we're looking at 8x.

Dell has officially announced a partnership with NVIDIA on constructing an AI factory that is designed to super-boost Grok training, the AI model created by Elon Musk's company, xAI. Dell CEO Michael Dell announced the news on a post on X, check it out below:

Grok is a generative AI chatbot created by Musk's xAI company, based on a large language model that process visual information like documents, diagrams, charts, screenshots, paragraphs and more, as well as its lengthy text capabilities.

SpaceX, Tesla, and xAI founder Elon Musk posted on X about the AI factory news, saying: "To be precise, Dell is assembling half the racks that are going into the supercomputer that xAI is building". In response to which other company xAI is working with, to which Musk replied "SMC" which is Super Micro Computer. Super Micro confirmed with Reuters that it's working with xAI.

Micron Technology is building new test production lines for advanced HBM in the US, while also considering manufacturing HBM chips in Malaysia for the first time, to capture more demand from the AI boom.

In a new report from Nikkei Asia and "sources briefed on the matter," the US-based memory chip company aims to triple its market share for HBM, one of the most crucial parts of AI chips, into the "mid-20" percentage range by 2025. There were reports on this just days ago, but it looks like Micron is full steam ahead now.

This is about the same level of Micron's market share for conventional dynamic random access memory (DRAM) chips, with TrendForce data showing Micron sits at around 23-25% market share of DRAM. Micron is also expanding its HBM-related research and development facilities, including production and verification lines, at its headquarters in Boisa, Idaho, USA, according to "two people with knowledge of the matter".

Foxconn has won exclusivity on manufacturing NVIDIA's new NVLink switches for its next-gen GB200 AI servers, a key component of GB200, which is known as the "magic weapon for improving computing power".

In a new report from UDN, we're learning that the order volume in 7x that of the GB200 AI server cabinets, where it's not just a new order, but the gross profit margin is also "much higher" than that of server assembly. It will become a powerful tool for Foxconn's gross profits and supplement its profits.

Foxconn has never commented on its orders or customer dynamics, but the industry has highlighted that the exclusive technology behind NVIDIA's new NVLink consists of two parts. First is the bridge technology, which connects the CPU and the AI chips together, while the other is the switch technology, which is the key to the interconnection between the GPUs.

NVIDIA couldn't be the company it is today without Taiwan, and CEO Jensen Huang knows that, and respects it, as well as the country itself.

During a recent interview, NVIDIA CEO Jensen Huang recently said that the company's $100 billion revenue is mostly manufactured in Taiwan, noting that AI has benefited Taiwan and that Taiwanese companies can thrive in the upcoming AI era.

In the full interview, Jensen said: said: "For over 25-years, we've been [working with companies in Taiwan]...Another reason is technology excellence. The ecosystem is unparalleled. Every aspect of building chips and building computers are located here. All the way down to the most minute, complex connectors that connect very high-speed cables are manufactured here".

NVIDIA announced the "most powerful AI chip on earth" by introducing its Blackwell-based GB200 AI superchip. Taiwan companies Foxconn and Quanta won major contracts for GB200 AI servers.

According to a new report by UDN, Foxconn will be handling 40% and Quanta will take 30% of GB200 AI servers, with shipments expected to begin in Q3 2024, driving a "big jump in overall revenue performance". NVIDIA's new DGX GB200 will also begin mass production in the second half of this year, with estimates of around 40,000 units in 2025.

Foxconn and Quanta will be the main suppliers of NVIDIA's upcoming NVL72 and NVL36 AI server cabinets, with UDN reporting that industry experts saying Foxconn has received orders for three categories of "DGX GB200" system cabinets (DGX NVL72, NLV32, and HGX B200). Foxconn said that GB200 series products will be shipped in rack form, with rough order estimates as high as 50,000 containers, with total revenue contribution to hit 2.5 trillion to 3 trillion yuan (around $344 billion to $413 billion USD or so).

NVIDIA CEO Jensen Huang made quite the famous trip to Taiwan this year, not just for Computex 2024, but to meet important tech partners in the country.

During his time, Jensen was interviewed by local media where he said that NVIDIA's annual growth exceeds $100 billion with various GPUs and AI GPUs, and that most of them are made in Taiwan. AI applications have benefitted Taiwan, and Taiwanese companies can thrive in the AI wave, reports CNA.

Jensen said that the AI era is a new beginning, something that NVIDIA has been preparing for a very long time now. Taiwan's technology industry is fully committed to the future of AI, with Jensen saying during a recent interview: "I am very happy to see this phenomenon".

Researchers have used machine learning techniques, the fundamental technology powering artificial intelligence, to analyze hundreds of wild elephants that were captured between 1986 and 2022.

The team published their findings in the peer-reviewed journal Nature Ecology and Evolution, which states that researchers looked at how each elephant "rumbles." These rumblings are believed to be the primary way elephants communicate with each other, and the study has broken the rumbles into three categories: greeting, caregiving, and contact calls.

While there are other prominent rumble categories, such as "let's go," the team found the aforementioned three types of rumble are most likely tied to names between elephants. Notably, the study found through the power of AI analysis that elephants appear to be attaching names to these rumbles, which is very different from imitation communication that is used by dolphins or parrots to communicate.

One of the most prolific tests a machine can undertake is called the Turing Test, created by Alan Turing in 1950 that's purpose was to test if a machine is capable of displaying a level of intelligence that can convince, or fool a human they are engaging with another human.

Researchers have put OpenAI's GPT-4 up against the Turing Test, and according to a new study the impressive large language model passed the test. The not-yet-peer-reviewed-study asked 500 people to engage with four different respondents. One of those respondents was a human, another was a 1960s-era AI called ELIZA, another was OpenAI's less sophisticated GPT-3.5, and finally GPT-4.

The paper states each conversation lasted approximately five minutes, and according to the study's results, humans found GPT-4 to be a human 54% of the time. These results lead researchers to claim GPT-4 has already passed the Turing Test. However, the study also indicated study participants believed the human respondent was a human 67% of the time, while ELIZA scored just 22%, and GPT-3.5 scored 50%.

After sustained criticism and widespread disdain, Microsoft will not roll out its controversial AI-powered 'screenshot everything you do on Windows 11, including capturing sensitive data' Recall feature as part of the Copilot+ PC launch. Instead, a revamped Recall feature will be available in the coming weeks as part of the Windows Insider Program (WIP).

Copilot+ PCs, which are basically laptops with CPUs equipped with at least a 40 TOPS NPU, are set to hit retail shelves this week. The Arm-based Snapdragon X Elite processor exclusively powers the first run, which, without Recall, will be limited to Microsoft's AI tools for photo and video editing, live captions, and Cocreator image generation. Plus, apps from third parties that require 40 TOPS of AI Performance.

This is for the best, as Recall has been under fire since its announcement on May 20. The controversial snapshot tool - designed to make it easier to search through your local PC usage to find documents, images, or anything else - was quickly discovered to be lacking security features. It would record things like passwords, banking information, and more.