Vast GPU Arsenal Required for Text-to-Video Revolution

The emergence of AI-generated videos has sparked a surge in GPU demand, with estimates suggesting a staggering 720,000 high-end Nvidia H100 GPUs are necessary to support the creator community on platforms like TikTok and YouTube. The training process for such models requires immense compute power, surpassing even GPT-4 and still image generation. As adoption grows, inference demands will further outpace training, necessitating a massive hardware investment to make text-to-video generation mainstream.

Text-to-Video Revolution Hinges on a Vast GPU Arsenal: Millions Required

The groundbreaking capabilities of text-to-video generators, exemplified by OpenAI's Sora, have sparked a surge of excitement and investment in Artificial Intelligence (AI). However, the realization of mainstream AI-generated video hinges on the availability of an unprecedented number of specialized processing units known as Graphics Processing Units (GPUs).

Estimates by Factorial Funds reveal that a staggering 720,000 high-end Nvidia H100 GPUs would be necessary to cater to the vast creator communities of platforms like TikTok and YouTube. The gargantuan scale of this computational demand is evident in Sora's training process, which requires up to 10,500 powerful GPUs for a month and can generate only about 5 minutes of video per hour per GPU during inference.

As illustrated by the data, training text-to-video models demands significantly more computing power than training massive language models like GPT4 or even generating static images. Moreover, as AI models like Sora gain wider adoption, the computation required for inference - the process of using the trained model to generate new videos - will eclipse the computation required for initial training.

To put this into perspective, Nvidia shipped approximately 550,000 H100 GPUs in 2023. Statistics from Statista indicate that the twelve largest customers utilizing Nvidia's H100 GPUs collectively possess 650,000 of these cards, with tech giants Meta and Microsoft accounting for 300,000 of them.

Assuming a cost of $30,000 per card, the astronomical sum of $21.6 billion would be required to procure the GPUs necessary to make Sora's vision of AI-generated text-to-video mainstream. This figure nearly matches the entire market capitalization of AI tokens at present.

The procurement of such a vast quantity of GPUs presents a formidable challenge due to supply constraints and escalating demand for AI-driven applications. Notably, Nvidia is not the sole player in the GPU market. Rival chipmaker AMD offers competitive products, with investors generously rewarding the company for its innovations, propelling its stock from the $2 range in 2012 to over $175 today.

Alternative options exist for outsourcing computing power to GPU farms, such as Render (RNDR) and Akash Network (AKT). However, the majority of GPUs on these networks consist of retail-grade gaming GPUs, significantly less potent than Nvidia's server-grade H100 or AMD's comparable offerings.

Despite these challenges, the tantalizing promise of text-to-video generation, as envisioned by Sora and other protocols, will necessitate a herculean hardware investment. While the potential for revolutionizing the creative workflow in industries like Hollywood is undeniable, it is crucial to temper expectations regarding the imminent mainstream adoption of AI-generated video.

At present, the industry faces a pressing need for a substantial increase in chip production to meet the burgeoning demand for AI-driven applications. Until this supply gap is addressed, the widespread availability of AI-generated video will remain an elusive goal.