By: Bill Pearson, vice president, Data Center Software, Intel Corporation, Anil Nanduri, vice president, AI Data Center Go-to-Market Strategy
Enterprise AI is moving beyond a narrow question: Which accelerator posts the highest benchmark number?
In production, a model is only one part of the system. A typical enterprise request may retrieve internal documents, validate permissions, search a vector database, call a business system, apply guardrails, generate a response, and record the interaction for audit and improvement. Inference matters—but so do the services around it.
That is why the next phase of enterprise AI will be shaped by software integration coupled with peak hardware performance. CIOs and engineering leaders are not deploying benchmarks. They are building reliable applications, managing costs, meeting security requirements, and creating a path from experimentation to production.
Meet Developers Where They Work
Developers should expect vendors to make their hardware as easy to adopt as possible. The most practical way to reduce friction is to support the frameworks and runtimes developers already use, and support them in the way they are used to.
That means contributing optimizations upstream in projects such as PyTorch, SGLang, and vLLM rather than asking developers to learn a proprietary language, rewrite kernels, or redesign applications around a vendor-specific toolkit.
A developer should be able to evaluate a model in a familiar framework, use standard deployment patterns, and move it into an existing environment. We describe that objective as Day 0 readiness: enabling relevant new models through the software ecosystems developers already depend on, with less integration work between a model release and a production workload.
Open-source frameworks are where much of today’s AI innovation happens. They offer visibility into the stack, make optimizations broadly available, and help enterprises avoid tying their long-term application architecture to a single vendor’s tools.
Production AI Is a Systems Problem
The future of enterprise AI is more than running queries on a single model. It is agentic software operating across a complex, asynchronous workflow, breaking down larger requests into smaller tasks, and running those tasks across different tools and services resulting in a single output. Intel has been helping organizations scale these types of enterprise and high-performance computing workloads for decades.
A production agent may retrieve private data through RAG, parse business documents, enforce fine-grained access controls, invoke ERP or ticketing systems, orchestrate external tools, and route sensitive actions to human review. Every stage introduces performance, security, and reliability requirements.
This can be an inherently a heterogeneous workload. Accelerators perform dense model computation, while CPUs govern the operational framework around the model: preparing and moving data, handling asynchronous requests, coordinating tool calls, enforcing guardrails, supporting low-latency networking, and connecting directly to enterprise business logic.
That division of labor is why Intel Xeon remains foundational to enterprise AI infrastructure. The CPU is not simply a host for an accelerator; it is the control plane that connects inference to the rest of the business.
Xeon’s software ecosystem has been proven across enterprise data centers for decades. Its mature memory architecture, hardware-rooted security, virtualization, and database performance allow organizations to deploy sophisticated agentic systems on infrastructure they already know and operate.
Build an Open GPU Stack
Intel’s discrete GPU software stack is newer than the Xeon ecosystem, and we are building it with that reality in mind.
The goal is to make Intel GPUs easily accessible through open standards and familiar tools—not to require an Intel-specific programming environment for common AI workflows. As we advance next-generation GPU platforms, including our forthcoming next-generation data center GPU, code-named Crescent Island, our focus is on three areas:
- Native framework integration. Developers should be able to code for Intel GPUs through standard frameworks and open inference engines, while the underlying work on kernels, execution graphs, attention mechanisms, and runtimes remains largely behind the scenes.
- Easier performance tuning. Strong accelerator performance can demand specialized knowledge. Tools such as Intel GPU AI Skills can help developers profile, tune, and deploy models through natural-language interaction, allowing more teams to begin with an informed configuration rather than mastering every platform detail first.
- Workload portability. Enterprises should not have to rewrite application logic when they decide a part of a workflow belongs on a CPU, GPU, or specialized accelerator. Software should make it easier to place work where it makes the most operational and economic sense.
This is not a one-time compatibility exercise. Customers need a stack that improves alongside the models, frameworks, and deployment patterns they adopt.
Make Disaggregated Inference Practical
As models grow and inference demand becomes less predictable, more organizations are considering disaggregated inference: separating prefill, which processes a prompt and its context, from decode, which generates tokens one at a time.
The two phases place different demands on compute and memory bandwidth. Running both on the same infrastructure can be simpler, but separating them may improve utilization at scale.
The hard part is not describing the architecture. It is operating it.
A real deployment must move KV-cache state efficiently, manage network latency, schedule requests across nodes, avoid bottlenecks during demand spikes, recover from constrained resources, and expose enough telemetry for operators to understand changes in latency or cost.
Software turns a collection of servers into an inference service. It routes work, manages state, provides observability, and helps operators use resources efficiently.
The rise of open models requires infrastructure which matches the right hardware to the right part of the workload. This is enabled by open runtimes and frameworks so customers can deploy these architectures using open software alone. The objective is choice: the flexibility to adopt the deployment model that fits a workload and change it later as requirements evolve.
From Model to Enterprise Service
Moving from a model checkpoint to a dependable business service requires three things:
- Optimize and serve. The model must run efficiently in the chosen framework and serving engine across throughput, latency, memory use, and cost—not just a single benchmark. Upstream work in frameworks and distributed serving engines gives developers a common starting point.
- Package for operations. Raw model weights are not an enterprise product. Teams need containerized services, deployment automation, health checks, logging, telemetry, scaling behavior, and an operational path for environments such as Kubernetes. Intel Inference Microservices are designed to provide modular building blocks for this transition.
- Connect data and controls. Models must interact safely with knowledge repositories, vector indexes, identity systems, APIs, policy layers, and human-review processes. Reference architectures, RAG and agent toolkits like Intel’s AI for Enterprise Agent toolkits, can help teams avoid rebuilding the same integration patterns for each project.
The Enterprise Choice
AI infrastructure decisions are becoming more practical. Enterprises are evaluating whether systems can sustain real workloads, integrate with existing operations, and remain flexible as models and requirements change.
Closed stacks can be convenient, particularly early in a deployment, however they can also limit portability, reduce control and transparency and slow innovation adoption.
Open, integrated software gives enterprises more options: use familiar frameworks, choose the hardware that fits each workload, and evolve architecture without starting over.
That is software’s strategic role in the agentic era. It connects silicon to enterprise systems, coordinates heterogeneous infrastructure, and turns model capability into reliable applications.
For Intel, the priority is clear: build and contribute to an open software foundation that lets customers use CPUs, GPUs, and other accelerators as parts of one production AI system—not as isolated products