✍️ Text Generation¶

🧪 ICML2026 · 2 paper notes

Characterizing the Effect of Noise in Language Generation in the Limit: Under the Kleinberg-Mullainathan formal framework of "language generation in the limit," this paper proves that for both uniform and non-uniform generation, noise level 1 is equivalent to any finite noise level \(i \geq 1\) (hierarchy collapse), while a strict separation exists between the noise-free case and noise level 1. Furthermore, it provides the first complete characterization of non-uniform noise-dependent generatability.
Score-Repellent Monte Carlo: Toward Efficient Non-Markovian Sampler with Constant Memory in General State Spaces: SRMC utilizes a \(d\)-dimensional running score average (rather than an \(|\mathcal{X}|\)-dimensional empirical measure) to record history. This history is then incorporated into an exponential score-tilt to construct a surrogate target \(\pi_\theta\) that "repels already visited regions." By wrapping this around any base MCMC kernel, the authors implement a non-Markovian, low-variance, normalization-free sampler with constant memory in general state spaces.