Beyond Prefill and Decode: Disaggregation Moves from Tokens to Frames

Author: Kiran Atmakuri

Coauthors: Pallavi Jaini, Brian Liu, Sergey Plotnikov and Daniel Socek

Up to 28% higher measured video-generation throughput at the same GPU count on Intel® Arc™ Pro B70 Graphics GPUs using Dynamo and vLLM-Omni.

The next opportunity in inference optimization is not simply a faster model. It is a better way to organize the work that model performs.

Our implementation applies this systems approach to streaming diffusion video. By separating latent generation from pixel decoding on Intel Arc Pro B70 Graphics GPUs, it delivers up to 28% higher measured video-generation throughput than the best aggregated configuration in our reported tests, at the same GPU count.

The architectural opportunity is to move beyond how efficiently we generate tokens to how efficiently we generate and deliver frames.

For language models, disaggregation separates prompt processing from autoregressive token generation. These phases place different demands on compute and memory, so separating them allows different execution and scaling strategies. Multimodal inference extends the principle to input encoding, allowing media processing to run independently of the language model. [1]

Streaming diffusion presents a different pipeline, but the same architectural question: Which stages should run together, and which benefit from independent resources?

The streaming video implementation separates the diffusion transformer (DiT) from the variational autoencoder (VAE) decoder. The DiT generates latents, the model’s compressed video representation, and the VAE converts those latents into RGB frames. Text encoding remains with the DiT; H.264 video encoding uses a media engine on an existing B70 GPU.

The boundary is therefore latent generation versus pixel decoding, not a division of successive diffusion denoising steps across GPUs. Nor does this example require different GPU architectures: independent allocation across B70 GPUs is enough to produce a measurable benefit.

Streaming changes where the bottleneck appears

AI video streaming models generate video incrementally, one latent block at a time, rather than producing an entire clip before any frames can be delivered. This requires a causal generation process in which each new block depends only on previously generated content rather than on future latents.

Conventional video diffusion models such as Alibaba’s Wan generate a sequence of video latents through multiple bidirectional denoising steps before decoding the completed latent sequence into pixels. Causal Forcing transforms this execution pattern for streaming generation. It distills the multi-step bidirectional model into a fast autoregressive student that can generate each new latent block causally using only previously generated content [1].

The distillation process serves two purposes. It reduces the number of denoising steps required for each generated block, and it trains the autoregressive student to remain stable as generation progresses. This helps mitigate error accumulation and exposure bias, which can otherwise lead to visual degradation during long autoregressive video generation.

In our implementation, the distilled model performs a one-step DiT rollout for each latent block. The VAE then immediately decodes that block into RGB frames before generation continues with the next block. Causal attention enables this incremental execution because the model no longer depends on future latent blocks.

This changes the systems architecture significantly. Instead of VAE decoding occurring primarily once at the end of video generation, DiT latent generation and VAE pixel decoding become recurring stages of the streaming pipeline. That repeated DiT-to-VAE handoff creates a natural boundary for disaggregation.

Different stages need different scaling strategies

The DiT can use replication and tensor parallelism. In this VAE implementation, however, the temporal cache is module-level state, so a worker decodes one session at a time rather than simply batching independent sessions together. The implementation accelerates VAE decoding through spatial sharding: dividing a latent by height across GPUs and exchanging the boundary rows needed for convolution.

These are different ways of using additional hardware. Separating the stages allows each to use the approach appropriate to its execution characteristics.

Scale the measured bottleneck—not every stage equally

Profiling shows that VAE decoding is the slower stage in this streaming pipeline. Spatially sharding VAE decoding across additional GPUs reduces that bottleneck and brings its execution rate closer to the DiT stage.

This motivates allocating additional GPU resources to VAE decoding while keeping DiT execution on a smaller GPU pool. The resource split follows the characteristics of each stage rather than requiring every stage to scale identically.

Separate memory budgets and overlap execution

DiT alone occupies approximately 25.9 GB of a 32 GB B70 GPU in the reported configuration. Separating the VAE and its activations creates more headroom for cache growth, encoding buffers, and other runtime allocations.

The split also enables a pipeline: DiT can generate latent block N+1 while VAE decodes block N. In steady state, the completion interval can approach the slower stage’s execution time rather than the sum of both stages, subject to transfer and synchronization overhead. This improves the cadence of successive outputs; it does not remove the dependencies that determine first-output latency. ^[1]

Together, these characteristics provide the case for diffusion disaggregation: independent scaling, separate memory budgets and overlapping execution.

To evaluate the benefit of DiT/VAE disaggregation, we compare aggregated and disaggregated configurations while keeping the total number of Intel Arc Pro B70 Graphics GPUs constant. This is important because it isolates the benefit of stage separation and independent resource allocation from the simpler effect of adding more hardware.

The engineering evaluation uses Causal Forcing Wan 2.1-1.3B T2V with a three-scene, 243-frame workload at 832 x 480 resolution. The software stack uses Dynamo with a vLLM-Omni backend, with H.264 encoding offloaded through VA-API on the B70 hardware media engine.

In the aggregated configuration, the same GPU pool is used for both DiT execution and VAE decoding. Scaling the configuration therefore couples the resources available to the two stages. In the disaggregated configuration, the DiT and VAE operate on separate GPU pools, allowing the available GPUs to be allocated according to the execution characteristics of each stage.

With two GPUs, the disaggregated configuration delivers 14% higher measured video-generation throughput than the best aggregated configuration at the same GPU count. With four GPUs, the advantage increases to 28%.

The important point is that these gains do not come from using more GPUs. The improvement comes from allowing the DiT and VAE stages to scale independently according to their workloads. By placing them on separate GPU pools, the disaggregated configuration avoids resource contention between the two stages and allows more of the available GPU capacity to be directed toward the stage that benefits most from additional resources.

The result is more performance from the same GPU count, not merely more performance from more hardware.

To make the architecture usable end to end, our implementation brings together four pieces of engineering work: porting the Causal Forcing Wan model to vLLM-Omni, decoupling DiT and VAE execution, enabling hardware video encoding and adding Common Media Application Format (CMAF) streaming support in the Dynamo orchestration framework.

The Dynamo frontend accepts HTTP requests and serves the fragmented-MP4 streaming endpoint, forwarding video fragments to the client as they are produced. The client submits an OpenAI-compatible request containing either a single prompt or a multi-scene storyboard, without needing to manage the placement of model stages.

The DiT worker performs text encoding and autoregressive latent rollout. The VAE workers convert those latents into RGB frames. In the illustrated three-way VAE configuration, the workers split each latent by height and exchange halo rows at each convolution.

The Omni router owns the request lifecycle and runs H.264 encoding and CMAF fragmentation through an FFmpeg child process. H.264 compression is offloaded to the B70 hardware media engine, while the surrounding software coordinates the encoding process, fragmentation and delivery.

This connects stage-level optimization to an application-facing service: requests enter through the frontend, model workers produce frames, and the streaming path delivers playable video fragments progressively.

The engineering implementation is provided in the cf_wan_pipe branches of the vLLM-Omni and Dynamo repositories in the References section [2]. These branches identify the implementation used for this work; they should not be read as a claim that every capability is included in a released upstream version.

Building on Intel’s work in disaggregated inference

This work extends Intel’s ongoing effort to make disaggregated inference practical through open software.

In July 2025, Optimizing LLM Inference on Intel® Gaudi® Accelerators with llm-d Decoupling demonstrated prefill/decode separation on Intel Gaudi accelerators with llm-d and vLLM. The work showed how the stages could be scheduled and scaled independently, with more consistent inter-token latency in the reported tests. ^[3]

Our March 2026 article, From Models to Systems: Enabling Heterogeneous AI Inference with Open Orchestration, broadened that discussion to prefill/decode and encode/prefill/decode pipelines, alongside Intel’s contributions to open orchestration through Dynamo. The focus was on matching resources to each stage’s execution needs. ^[4]

In June 2026, Lowering Multimodal Inference Cost with Heterogeneous E/PD Disaggregation extended the same idea to heterogeneous multimodal inference, showing how independently placing encode and prefill/decode stages on different accelerator resources can improve inference efficiency and economics. ^[5]

This diffusion implementation carries the same principle into a new workload: DiT latent generation and VAE pixel decoding on Intel Arc Pro B70 Graphics GPUs, connected through Dynamo and vLLM-Omni. The progression is from token-serving stages to frame-generating stages, with the same goal: independently optimize the work that determines the service’s performance.

Architecture + B70 + open software = customer value

The value of disaggregation is not separation for its own sake. It is the ability to allocate resources where they improve the customer’s service.

Here, the pieces reinforce one another. DiT/VAE disaggregation allows independent stage scaling and overlapping execution. Intel Arc Pro B70 Graphics GPUs provide the compute, memory and hardware video encoding used by the pipeline. Dynamo and vLLM-Omni connect those components into an end-to-end streaming implementation.

Together, they deliver up to 28% higher measured video-generation throughput at the same GPU count in the reported comparisons. That means more video-generation throughput for an equivalent GPU investment, assuming the same GPU acquisition cost.

This is a concrete step toward better inference economics. Quantifying total cost of ownership still requires power, host and infrastructure costs, utilization and service-quality measurements. The measured throughput advantage is not an equivalent measured reduction in TCO.

From tokens to frames, the principle remains the same: scale the work that matters, not every stage equally.