Optimus: Elastic Decoding for Efficient Diffusion LLM Serving

AIPR assessment

Problem difficulty is high: this sits in a competitive, well-optimized serving space where small changes in granularity and scheduling can materially affect throughput and latency. The strongest aspects reinforce each other, the method is conceptually simple, the hardware evaluation is real, the gains are consistent across workloads, and the open-source implementation makes the result usable. The main weaknesses also interact, the runtime token-efficiency model is heuristic, the comparison set i

Abstract

Large language model (LLM) serving is fundamentally limited by inefficient hardware utilization. Autoregressive (AR) decoding underutilizes GPUs due to its strictly sequential execution, while diffusion LLMs (DLLMs) improve throughput by decoding multiple tokens per iteration. However, fixed block-size diffusion decoding exhibits strong load sensitivity: large blocks exploit idle GPU resources under low load, but saturate early and incur substantial redundant computation under high load. As a result, throughput gains vanish beyond saturation, and no single decoding granularity performs well across dynamic serving workloads. We present Optimus, a serving system that enables elastic decoding for diffusion LLMs by dynamically adapting decoding granularity to runtime load. The key idea is to treat decoding granularity as a runtime control variable, balancing GPU utilization and token efficiency. Optimus combines chunked decoding, which enables fine-grained execution without retraining, with saturation-aware scheduling, a closed-loop mechanism that selects chunk sizes based on runtime conditions. Together with system-level optimizations and customized attention kernels, Optimus achieves significant performance improvements while preserving model accuracy. Experiments show that Optimus delivers up to 6.1x throughput improvement over AR decoding and 4.3x improvement over fixed-block diffusion LLM, while maintaining stable performance across diverse load regimes and improving end-to-end serving capacity under latency constraints. The source code is available at https://github.com/dubcyfor3/Optimus.

Score Breakdown

More from this week

• TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
• Context Features Are Cheap: Rank-Aware Decomposition for Efficient Feature Interaction in Recommender Systems
• Rethinking Continual Anomaly Detection on the Edge: Benchmarking Under Realistic Industrial Conditions
• Learning High-Frequency Continuous Action Chunks in Latent Space
• A Pragmatic Approach to Learned Indexing in RocksDB: Targeted Optimizations with Minimal System Modification

More in Computer Architecture

• ScaleDisturb: Exploiting Temporal Asymmetry to Amplify Read Disturbance in Modern DRAM Chips
• ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving
• DORA: Dataflow-Instruction Orchestration Architecture for DNN Acceleration
• VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
• Loaded Dice: Solving the Non-Selection Problem for Scalable Probabilistic RowHammer Defense