Intel has launched its Gaudi 3 AI accelerator, aiming to compete with NVIDIA’s dominant H100 and H200 GPUs by offering a cheaper alternative with a lower total cost of ownership (TCO). While Gaudi 3 falls behind in raw performance, it boasts features optimized for large-scale generative AI and promises seamless integration with popular frameworks like PyTorch and Hugging Face.
NVIDIA’s Hopper H100 and H200 AI GPUs are getting even more powerful with new optimizations in the CUDA stack. The H200, featuring 80% more HBM memory and 40% higher bandwidth than the H100, delivers significant performance gains across a range of benchmarks, including Llama 2 and Mixtral 8x7B. These advancements highlight NVIDIA’s commitment to pushing the boundaries of AI computing.
Meta’s Llama 3 405B model training took 54 days on a cluster of 16,384 NVIDIA H100 AI GPUs, facing numerous hardware challenges. The team overcame these hurdles with in-house tools and efficient strategies, achieving a 90% effective training time.
Elon Musk, CEO of Tesla, expressed his dissatisfaction with the term “GPU” during the company’s first-quarter earnings call. He stated that Tesla’s core AI infrastructure is no longer training-constrained, with the company actively expanding its infrastructure. Tesla has installed 35,000 H100 computers or GPUs and anticipates reaching 85,000 by year-end. Despite the revenue miss in the first quarter, Musk emphasized the efficient use of H100s. Nvidia’s H100 chip continues to experience high demand, with customers facing wait times due to supply constraints.
