Session

DescriptionMinimizing inference compute resources for modern LLM serving is limited by a token-time problem: time-to-first-token and tokens-per-second are increasingly dictated by precision choices, expert routing, and KV-cache traffic. This session features new low-bit quantization/formats, MoE acceleration (including speculative decoding and pruning), and KV-cache management/retrieval for long-context and streaming workloads. Together, the papers deliver a clear blueprint for faster and cheaper LLM inference in both interactive and batched settings.