✍️ Text Generation

🔬 ICLR2026 · 12 paper notes

📌 Same area in other venues: 💬 ACL2026 (17) · 🧪 ICML2026 (2) · 🤖 AAAI2026 (3) · 📹 ICCV2025 (1) · 🧪 ICML2025 (1) · 💬 ACL2025 (27)

🔥 Top topics: Diffusion Models ×4 · LLM ×2

Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models

Antislop treats "AI-typical repetitive phrases (slop) in LLM generation" as quantifiable, locatable, and erasable objects. It first maps model-specific "slop fingerprints" using frequency ratio statistics, then utilizes an inference-time backtracking sampler to precisely suppress these patterns. Finally, it automatically converts the sampler's interception records into preference data for the newly proposed FTPO fine-tuning, permanently welding the suppression capability into the weights—achieving a 90% reduction in slop with almost no performance degradation on GSM8K/MMLU/Creative Writing.

Causal-Steer: Disentangled Continuous Style Control without Parallel Corpora

This paper proposes Causal-Steer: by treating LoRA as a "causal intervention," it computes the difference in activations with and without LoRA perturbations on the same input. This approach eliminates the need for parallel corpora and extracts a clean style vector. After PCA denoising and robust aggregation via geometric median, it achieves continuous, bidirectional, and linearly interpolatable style control for LLMs using a single scalar \(\alpha\) during inference.

Diverse Text Decoding via Iterative Reweighting

This paper proposes OverRIDE (Reweighting-based Iterative DEcoding), which incrementally fine-tunes a "guidance model" using historical generated results during multi-round sampling. By suppressing the probabilities of tokens that lead to historical pattern recurrence, it significantly enhances diversity across multiple responses with minimal quality loss and can be integrated into serving systems like vLLM with only 6.4% throughput loss (72B).

FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Model

Proposes FS-DFM (Few-Step Discrete Flow-Matching), which reduces the sampling steps of discrete flow-matching language models from 1024 to 8 via step-aware training and a cumulative scalar update rule, achieving 128x acceleration while maintaining comparable perplexity and generation quality.

Improving Attributed Long-form Question Answering with Intent Awareness

Addressing the issues of "poor citation quality and low readability" in long-form reports generated by deep research systems, this paper proposes a tag-based dual-layer intent (paragraph intent + citation intent) writing framework. This framework enhances Large Language Models (LLMs) via direct prompting during inference and distills Small Language Models (SLMs) using intent-aware synthetic data. Across three scientific report generation benchmarks, LLMs improved by an average of +2.9 points and SLMs by +12.3 points, with citation metrics showing particularly significant gains.

Logit-KL Flow Matching: Non-Autoregressive Text Generation with Sampling-Mixing Inference

Ours uses "linear interpolation in logit space" (equivalent to the KL geodesic on the simplex) as the path for discrete flow matching. It proves that maximizing conditional likelihood exactly recovers the velocity field and introduces a "denoise-and-renoise" iterative sampler and hybrid inference scheme, significantly reducing perplexity and improving BLEU for non-autoregressive text/code generation.

p-less Sampling: A Robust Hyperparameter-Free Approach for LLM Decoding

This paper proposes p-less sampling: a completely hyperparameter-free truncation decoding method. At each step, it uses the "collision probability" \(\sum_v P_\theta(v)^2\) of the entire token distribution as a dynamic truncation threshold. It outperforms methods like top-p and min-p in mathematics, logical reasoning, and creative writing, showing minimal degradation at high temperatures while offering faster inference.

Planner Aware Path Learning in Diffusion Language Models Training

This paper points out the mismatch between the default "random unmasking paths" used in training and the actual "planner-guided paths" used during inference in masked diffusion language models. It proposes Planner-Aware Path Learning (PAPL), which reweights the masked diffusion loss using planner confidence to align training more closely with the inference path. This leads to steady quality improvements in protein sequence, text, and code generation.

Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs

This paper identifies a persistent " <eos> overflow" early termination issue in instruction-tuned diffusion language models—where longer allocated generation lengths lead to shorter or even collapsed answers (sequences of <eos>). The root cause is that <eos> serves as both a terminator and a padding token. The authors propose Rainbow Padding: retaining a single <eos> for actual termination while filling remaining positions with a deterministic cycle of \(K\) distinct padding tokens. Using only 7 tokens and single-epoch LoRA, it restores length robustness, improving LLaDA's accuracy on MATH from 0.6% to 32.6%.

Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure

Starting from the framework of proper scoring rules, this work proves that the negative log-likelihood of the maximum probability output sequence (MSP) is a theoretically sound uncertainty measure. It proposes G-NLL—which approximates this measure using a single greedy decoding pass—matching or exceeding state-of-the-art (SOTA) sampling-based methods across multiple scenarios.

Text Summarization via Global Structure Awareness

GloSA-sum introduces Topological Data Analysis (TDA) to text summarization for the first time. It utilizes persistent homology to identify the document's semantic skeleton and logical loops, storing them in a "protection pool." A lightweight proxy metric is then used to iteratively delete sentences, achieving fast and accurate compression without losing core logic chains, while effectively shortening contexts for downstream LLM tasks.

Unveiling the Potential of Diffusion Large Language Model in Controllable Generation

This paper proposes Self-adaptive Schema Scaffolding (S3)—a training-free method that injects a structure template (schema) as a "semi-denoised initial state" directly into the output context of a Diffusion Large Language Model (dLLM). Augmented with null placeholders for adaptive length, S3 allows dLLMs to stably generate valid structured outputs like JSON with fewer denoising steps. It improves structure compliance from the 30%–80% baseline range to over 99%, while maintaining a lower hallucination rate.