OpenAI Sora video tool large-scale deployment uses 720,000 NVIDIA H100 GPUs worth $21.6 billion

OpenAI video tool 'Sora' compute estimates for large-scale deployment: 720,000 x NVIDIA H100 AI GPUs, worth over $21 billion in total.

2 minutes & 56 seconds read time

OpenAI's impressive new text-to-video tool, Sora, loves some GPU compute power. New numbers from Factorial Funds estimate that 720,000 x NVIDIA H100 AI GPUs would be needed for peak times on Sora.

AI inference compute comparison (source: Factorial Funds)

AI inference compute comparison (source: Factorial Funds)

720,000 x NVIDIA AI GPUs is a monumental amount of AI GPU computing power. With each costing around $30,000 x 720,000, that's $21.6 billion. Not only is it a mountain of money, but the amount of power at 700W per GPU is astounding, too, totaling 504,000,000W of power. Yeah, that's a lot of power.

Factorial Funds estimated that Sora used between 4,200 and 10,500 NVIDIA H100 AI GPUs for one month, with a single H100 AI GPU capable of generating a one-minute video in about 12 minutes, or around 5 x one-minute videos per hour.

NVIDIA's record-breaking revenue pushing the company over a $2.1 trillion market cap, as well as completely dominating the AI GPU market with 90%+ of the AI GPU market share powered by NVIDIA. OpenAI's new Sora text-to-video tool will be used by some of the world's biggest companies and people, so AI GPU demand will only skyrocket from here.

I would love to see a breakdown of NVIDIA's new Blackwell B200 AI GPU powering Sora videos, as it represents a gigantic leap in AI performance over Hopper H100.

Factorial Funds writes that if the inference compute in Sora-like models that achieve significant market share will see around 5 minutes of video generated per NVIDIA H100 AI GPU per hour "would be needed to run Sora-like models at significant scale, meaning that AI-generated videos achieve a significant market penetration on popular video platforms like TikTok and YouTube.

  • We assume 5 minutes of videos produced per NVIDIA H100 per hour (see above for details), equivalent to 120 minutes of videos per H100 per day
  • TikTok: 17M minutes videos per day (34M total videos × avg. length of 30s), assuming 50% penetration by AI (source)
  • YouTube: 43M minutes videos per day, assuming 15% penetration by AI (mostly video below 2 min)
  • Total videos produced daily by AI: 8.5M + 6.5M = 10.7M minutes
  • Total NVIDIA H100 needed to support the creator community on TikTok & YouTube: 10.7M / 120 ≈ 89k

The site continues, saying that this figure is "likely too low due to various factors that need to be accounted for:

  • We assume 100% FLOPS utilization and do not consider memory and communication bottlenecks. In reality a utilization of 50% is more realistic, which adds a factor of 2x.
  • Demand is not distributed equally across time but instead is bursty. Peak demand is especially problematic since you need proprotionally more GPUs to still serve all traffic. We think that peak demand adds another factor of 2x for the maximum number of GPUs needed.
  • Creators will likely generate multiple candidate videos to select the best one from these candidates. We make the conservative assumption that on average 2 candidates for each uploaded video are generated, which adds another factor of 2x.
  • In total this leaves us with ~720k NVIDIA H100 GPUs at peak
Buy at Amazon

NVIDIA H100 80 GB Graphic Card PCIe HBM2e Memory 350W

TodayYesterday7 days ago30 days ago
Buy at Newegg
* Prices last scanned on 4/20/2024 at 4:38 am CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.

Anthony joined the TweakTown team in 2010 and has since reviewed 100s of graphics cards. Anthony is a long time PC enthusiast with a passion of hate for games built around consoles. FPS gaming since the pre-Quake days, where you were insulted if you used a mouse to aim, he has been addicted to gaming and hardware ever since. Working in IT retail for 10 years gave him great experience with custom-built PCs. His addiction to GPU tech is unwavering and has recently taken a keen interest in artificial intelligence (AI) hardware.

Newsletter Subscription

Related Tags