✍️ Text Generation¶

🧪 ICML2026 · 2 paper notes

Characterizing the Effect of Noise in Language Generation in the Limit: Under the Kleinberg-Mullainathan "language generation in the limit" formal framework, this paper proves that for uniform and non-uniform generation, noise level 1 is equivalent to any finite noise level \(i \geq 1\) (hierarchy collapse). However, a strict separation exists between noiseless and noise level 1, and the authors provide the first complete characterization of non-uniform noise-dependent generatability.
Score-Repellent Monte Carlo: Toward Efficient Non-Markovian Sampler with Constant Memory in General State Spaces: SRMC uses a \(d\)-dimensional running score average (rather than an \(|\mathcal{X}|\)-dimensional empirical measure) to record history. This history is transformed via exponential score-tilt into a surrogate target \(\pi_\theta\) that "repels already visited regions." By wrapping this around any base MCMC kernel, SRMC achieves a non-Markovian, low-variance, normalization-free sampler with constant memory in general state spaces.