Intel Delivers Scalable AI Performance in MLPerf Inference v6.0

Intel is demonstrating scalable artificial intelligence performance with results from the newly released MLPerf Inference v6.0 benchmarks, showcasing its Xeon 6 processors and Arc Pro B-Series GPUs for workstations, datacenters, and edge systems. The benchmarks reveal a four GPU system utilizing Intel Arc Pro B70 and B65 graphics delivers 128GB of VRAM, capable of running 120 billion parameter models, with the Arc Pro B70 achieving up to 1.8 times higher inference performance than the Arc Pro B60. Software optimizations within an open, containerized stack also improve performance, yielding up to 1.18 times higher gains on existing Intel Arc Pro B60 hardware compared to MLPerf v5.1. Anil Nanduri, Intel vice president, AI Products and GTM, Intel Data Center Group, said, “The combination of Intel Xeon 6 and Intel’s Arc Pro B-Series GPUs represent our investment to expand customer choice and value, offering real-world solutions that address both LLM models as well as traditional machine learning workloads, with leading performance and incredible value for graphics professionals and AI developers worldwide.” Intel Xeon 6 and Arc Pro B-Series GPU Performance in MLPerf v6.0 These systems are designed to cater to high-end workstations, data centers, and edge computing applications, offering a versatile solution for AI workloads. The architecture of the Intel Arc Pro B70 allows it to handle substantially larger models and context windows in multi-GPU configurations, providing up to 1.6 times the KV cache capacity when processing extensive models. Intel remains the sole server processor vendor submitting stand-alone CPU results for MLPerf inference benchmarks, demonstrating its commitment to advancing AI inference across diverse platforms, with Xeon 6 processors delivering up to a 1.9x generational performance gain in MLPerf Inference v5.1. 128GB VRAM Enables 120B Parameter Model Inference The increasing demand for capable artificial intelligence models is rapidly reshaping hardware requirements, with memory capacity emerging as a critical bottleneck. Systems now require substantial video random access memory (VRAM) to effectively run large language models. This increase in capacity allows for more complex AI tasks and larger datasets to be processed directly on the GPU, reducing reliance on slower system memory. Intel’s advancements extend beyond raw memory; the company has focused on optimizing the entire system for AI workloads. CPU-Accelerated System Performance & Intel’s AMX/AVX512 Technologies Intel is demonstrating a commitment to holistic AI system performance, extending beyond GPU throughput to emphasize the crucial role of the central processing unit. The CPU manages critical functions like memory management, task orchestration, and workload distribution, while maintaining security and operational continuity for modern AI infrastructure. Built-in AI acceleration technologies, including AMX and AVX512, enable efficient execution of large language model inference, fine-tuning, and classical machine learning tasks without requiring dedicated accelerator hardware. These advancements are particularly relevant as the professional compute market transitions, with creators and developers seeking performance and value without compromising data privacy or incurring subscription costs. Intel’s systems aim to provide an all-in-one inference platform with validated hardware and software, simplifying adoption and enhancing usability through containerized solutions optimized for Linux environments. The combination of Intel Xeon 6 and Intel’s Arc Pro B-Series GPUs represent our investment to expand customer choice and value, offering real-world solutions that address both LLM models as well as traditional machine learning workloads, with leading performance and incredible value for graphics professionals and AI developers worldwide. Anil Nanduri, Intel vice president, AI Products and GTM, Intel Data Center Group Source: https://newsroom.intel.com/artificial-intelligence/intel-delivers-ai-performance-mlperf-inference-v6-0#:~:text=2-,Intel%20Delivers%20Open%2C%20Scalable%20AI%20Performance%20in%20MLPerf%20Inference%20v6,for%20workstations%20and%20edge%20systems. AI INFERENCE ARC PRO GPUS INTEL MLPERF XEON 6