arXivPaperNeeds Review

Rollplex: Cross-Phase GPU Spatial Sharing for Vision Language Model Post-Training

Rollplex overlaps VLM prefix work with rollout decoding and shares weights across training phases. On 32 H800 GPUs, it reports higher throughput without changing synchronous on-policy updates.

arXiv · Aug 14, 2026

Open Source Open Markdown Open JSON

Source Summary

Rollplex moves response-independent prefix computation into the rollout decode window. Phase-aware HBM management and parallelism-aware weight sharing address a naive Qwen2.5-VL-32B footprint of roughly **165 GiB per GPU**.

Practical Implication

Teams post-training vision-language agents can inspect phase boundaries for independent prefix work and share compatible tensor storage across differing parallel layouts. The reported schedule preserves synchronous on-policy updates under the same GPU budget.

Agent-Ready Context

Rollplex moves response-independent prefix computation into the rollout decode window. Phase-aware HBM management and parallelism-aware weight sharing address a naive Qwen2.5-VL-32B footprint of roughly **165 GiB per GPU**.

Teams post-training vision-language agents can inspect phase boundaries for independent prefix work and share compatible tensor storage across differing parallel layouts. The reported schedule preserves synchronous on-policy updates under the same GPU budget.

On **32 H800 GPUs**, Rollplex reports **1.23×–1.30×** speedup over serial colocation and **1.57×–2.24×** over disaggregation. The material provides results for this hardware and workload setup, not broader cost or scaling evidence.

Connected Context · Feed7 Judgment

Rollplex adds a concrete systems pattern for visual on-policy post-training: overlap response-independent prefix work with rollout decoding and share compatible tensor storage across phase-specific parallel layouts. It strengthens the case that phase scheduling can outperform both serial colocation and disaggregation under a fixed cluster budget, while narrowing the evidence to synchronous Qwen2.5-VL training on 32 H800s rather than general scaling or cost efficiency.

Taking Reinforcement Learning Cross Datacenter — Nan Jiang, ModalBoth reorganize rollout infrastructure, but at different boundaries: Rollplex spatially shares one GPU cluster while preserving synchronous updates, whereas the cross-datacenter proposal separates rollout workers and synchronizes sparse weight deltas.Accelerating Transformers Fine-Tuning with NVIDIA NeMo AutoModelBoth show that post-training throughput can improve through infrastructure changes without changing the model objective, but NeMo AutoModel addresses MoE fine-tuning APIs while Rollplex targets phase overlap and memory sharing in VLM rollouts, so their speedups are not directly comparable.OPD-V: Visual On-Policy Self-Distillation with Modality BalanceOPD-V supplies an example of visual on-policy post-training whose rollout workload could benefit from Rollplex’s scheduling pattern; Rollplex addresses execution efficiency rather than OPD-V’s modality-balance objective.

Context Map

infraimagevideo

Uncertainty

On **32 H800 GPUs**, Rollplex reports **1.23×–1.30×** speedup over serial colocation and **1.57×–2.24×** over disaggregation. The material provides results for this hardware and workload setup, not broader cost or scaling evidence.