โก LLM Efficiency¶
๐ฌ EMNLP2026 ยท 5 paper notes
๐ Same area in other venues: ๐ง NeurIPS2026 (79) ยท ๐ท CVPR2026 (8) ยท ๐ฌ ICLR2026 (171) ยท ๐ฌ ACL2026 (23) ยท ๐งช ICML2026 (48) ยท ๐ค AAAI2026 (9)
๐ฅ Top topics: LLM ร2 ยท Reasoning ร2
- Compressing Sequences in the Latent Embedding Space: K-Token Merging for Large Language Models
-
This paper proposes K-Token Merging, a latent-space soft prompt compression method that fuses contiguous blocks of \(K\) token embeddings into a single compressed embedding via an average-initialized residual MLP with LoRA fine-tuning, achieving up to 75% prompt length reduction and a 27.8% throughput speedup with negligible accuracy degradation.
- Efficient Iterative Retrieval with Heterogeneous Batching
-
Addressing the complementary nature of compute-bound embedding and memory-bound generation in iterative RAG, ORTHRUS introduces a unified inference loop with heterogeneous batching, chunked embedding with incremental pooling, and workload-aware intra-batch scheduling to eliminate static GPU partitioning bubbles, achieving 1.28รโ4.52ร higher throughput and cutting end-to-end p99 latency by up to 55.8%.
- SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation
-
Addressing the core tension between the reasoning capacity deficit of on-device models and the privacy risks of cloud-centralized LLMs in personalized generation, SpecSteer introduces an asymmetric collaborative speculative decoding framework where an on-device private specialist drafts candidate sequences, a cloud generalist conducts non-private ratio-based verification, and a steering recovery mechanism corrects rejected tokens, closing the reasoning gap while delivering a 2.36ร inference speedup.
- Spexis: Speculative Lookahead Scheduling for LLM Inference
-
Spexis introduces speculative parallelism (SP) as an orthogonal scheduling dimension for multi-GPU LLM inference that overlaps self-speculative drafting with downstream pipeline verification while sharing the same KV cache, augmented with confidence-based selective speculation and lookahead remaining-length prediction to achieve up to 34% serving throughput speedup over optimized TP+PP baselines.
- Survival-Guided Length Control for Efficient Reasoning
-
This paper proposes a plug-in, training-free survival-guided length predictor for masked diffusion language models (DLMs) that recasts sequence termination as a discrete-time survival process over the end-of-sequence token, deriving a closed-form expected length from single-pass EOS probabilities and delivering up to 7ร decoding speedups across reasoning and code benchmarks with zero accuracy loss.