- Intel has launched the Gaudi 3 AI accelerator, challenging Nvidia’s H100.
- Gaudi 3 promises faster performance and lower costs for AI tasks.
- Boasting impressive benchmarks, Gaudi 3 aims to offer a compelling alternative in the AI accelerator market.
Intel has just unveiled its latest AI accelerator, the Gaudi 3, at the Intel Vision 2024 event. This new accelerator is set to compete directly with Nvidia’s H100, promising faster performance and lower costs for AI tasks.
According to Intel, the Gaudi 3 offers significant improvements over its predecessor, the Gaudi 2. It boasts a 4x increase in AI computing power for BF16 tasks, 1.5x more memory bandwidth, and 2x greater networking bandwidth. These enhancements make the Gaudi 3 a powerhouse for AI applications.
One of the key features of the Gaudi 3 is its manufacturing process. Built on a 5nm process, it incorporates 64 AI-custom and programmable TPCs (Tensor Processing Cores) and eight MMEs (Matrix Multiply Engines), capable of handling 64,000 parallel operations.
With 128GB of memory, 3.7TB of memory bandwidth, and 96MB of on-board SRAM, the Gaudi 3 is equipped to efficiently process large datasets.
Intel claims that the Gaudi 3 outperforms Nvidia’s H100 in various tasks, including training time, inference throughput, and power efficiency. For example, it boasts a 50% faster training time on models like Llama 7B and 13B, as well as GPT-3 175B models.
Additionally, it offers a 50% increase in inference throughput and 40% greater power efficiency on certain models.
To support flexible system scaling, the Gaudi 3 features 24 integrated 200Gb Ethernet ports, enabling easy connectivity and clustering of accelerators.
This approach differs from Nvidia‘s B200 NVL72 systems, as Intel opts for fast Ethernet connections between accelerators.
Justin Hotard, an executive at Intel, emphasized the importance of providing customers with more choices in the AI market. He stated, “Enterprises weigh considerations such as availability, scalability, performance, cost, and energy efficiency.
Intel Gaudi 3 stands out as the GenAI alternative presenting a compelling combination of price performance, system scalability, and time-to-value advantage.”
While Intel’s benchmarks highlight the superiority of the Gaudi 3 over Nvidia’s H100, it’s essential to approach these claims with a critical eye. As with any product comparison, real-world performance may vary depending on specific use cases and configurations.
Intel’s Gaudi 3 represents a significant advancement in AI acceleration technology. With its impressive performance metrics and competitive pricing, it poses a strong challenge to Nvidia’s dominance in the AI accelerator market.
However, the true test will be how it performs in real-world applications and whether it can keep pace with Nvidia’s future innovations, such as the rumored H200.
