Skip to content

💬 LLM (Other)

💬 EMNLP2026 · 49 paper notes

📌 Same area in other venues: 🧠 NeurIPS2026 (98) · 🎞️ ECCV2026 (13) · 📷 CVPR2026 (3) · 🔬 ICLR2026 (55) · 💬 ACL2026 (61) · 🧪 ICML2026 (39)

🔥 Top topics: LLM ×14 · Diffusion Models ×7 · Reasoning ×3 · Personalized Generation ×3 · Alignment/RLHF ×3

Jasper-Flash: Elastic Token Compression and Progressive Distillation for Inference-Scalable Text Embedding Models

Jasper-Flash proposes an inference-scalable text embedding framework integrating Elastic Token Compression (ETC) and Compression-Adaptive Progressive Distillation (CAPD), enabling dynamic sequence reduction on the fly; its 600M model slashes 1024-token encoding latency from 24.24 ms to 4.48 ms (a 5.4x speedup) while achieving an MTEB score of 74.75, outperforming substantially larger 4B models.

Muon^2: Boosting Muon via Adaptive Second-Moment Preconditioning

Addressing the severe computational and communication overhead in MUON caused by the ill-conditioned momentum spectrum during Newton-Schulz iterations, MUON2 applies Adam-style adaptive second-moment preconditioning prior to orthogonalization, achieving superior directional alignment with 40% fewer iterations and cutting pretraining wall-clock time by 25%.

TCC-Bench: Benchmarking the Traditional Chinese Culture Understanding Capabilities of MLLMs

To assess the vulnerabilities of multimodal large language models (MLLMs) in non-Western cultural understanding, this paper introduces TCC-Bench, the first comprehensive bilingual VQA benchmark spanning eight distinct knowledge domains with 770 curated images and 1,015 questions, revealing persistent deficits across both open- and closed-source models when grounding nuanced cultural knowledge in fine-grained visual evidence.

Can LLMs Reliably Self-Report Adversarial Prefills, and How?

Across 10 open-weight instruction-tuned LLMs and four safety benchmarks, models fail to reliably self-report whether a previous response was elicited by an adversarial prefill (claiming intent on 25.3% of compromised responses); this introspective signal is causally mediated by the internal refusal direction, and targeted finetuning to flag compromised responses counterintuitively increases initial attack success rates due to cross-turn parameter sharing.

TRACEPROBE: An Execution-Based Benchmark for Diagnosing Safety Failures in LLM Agents

TRACEPROBE is an execution-based sandbox benchmark for diagnosing LLM agent safety across 155 scenarios and three risk sources, using normalized traces and deterministic action replay to localize intermediate latent violations and pinpoint the exact step of failure.

ABDATABENCH: Evaluating AI Agents for Evidence-Grounded Antibody Record Construction

Addressing the bottleneck of dispersed and unstructured scientific evidence in therapeutic antibody literature, this paper presents ABDATABENCH, the first benchmark evaluating complete 22-field antibody record construction across 44 expert-annotated documents (279 records), alongside ABCURATOR, a record-centric multi-agent framework paired with an expert-driven skill accumulation workflow.

Complexity-Aware Routing: Detecting and Allocating Dynamic Compute for Efficient Mixture-of-Experts Large Language Models

CAR introduces an information-theoretic dual-signal framework combining Shannon Surprisal and Predictive Entropy within a decoupled perception-decision architecture, enabling dynamic expert allocation that cuts active parameters by ~10% while outperforming static Top-k and existing dynamic routing baselines.

Sampling Less, Knowing More: Contradiction-aware Adaptive Spectral and Entropic Uncertainty Quantification for LLMs

The CASE framework constructs a sparsified bidirectional contradiction graph over sampled LLM candidate responses and computes its normalized spectral norm to quantify semantic disagreement, enabling dynamic early-stopping sampling that cuts generation costs by over 30% while combining token-level predictive entropy from a deterministic anchor to resolve failures on consistent errors.

Syntactic Complexity Convergence in Dialogue: Analysis of the Phenomenon and Application to LLM Detection

This paper conducts a systematic investigation of syntactic complexity convergence across full-length dialogues in three languages, discovering that while human dyads exhibit significant within-metric convergence in task-oriented settings, human-LLM dialogues exhibit a unique cross-metric convergence revealed by Linear-CKA, which enables an estimator-free Fourier-based detection of LLM-involved dialogues achieving 96% accuracy.

Advancing Fact-Checking for Large Language Models via Argumentative Reasoning

Addressing confirmation bias in retrieval and multi-source evidence conflict in real-world claim verification, this paper introduces ArgCheck, a neuro-symbolic framework coupling multi-round adversarial Pro-Con evidence collection with formal Dung abstract argumentation under grounded semantics, improving Macro-F1 by 5.39 and 11.10 points on MAVEN and AVeriTeC, respectively.

ClaimFlow: Tracing the Evolution of Scientific Claims in NLP

This paper introduces ClaimFlow, the first claim-centric benchmark and dataset tracing cross-paper scientific interactions across 1,617 expert-annotated NLP papers, formalizes the Claim Relation Classification task, and constructs an automated evolution graph spanning ~13k ACL Anthology papers to quantitatively reveal how scientific ideas propagate, face early vetting, and mature primarily through extension and qualification.

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think

CoDAR factorizes text generation into a continuous diffusion process within a compact embedding space and a contextual autoregressive Transformer decoder with cross-attention, resolving the longstanding token rounding bottleneck and achieving a 1.98x wall-clock speedup over discrete diffusion baselines.

Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation

Addressing the issue where sentence-level citations in RAG are frequently cluttered with irrelevant clauses while isolated keywords lack necessary referential antecedents, this paper defines a concise and sufficient sub-sentence citation task, constructs a 2,000-instance benchmark with 95% inter-annotator agreement, and proposes a dual-stage filtered data augmentation and two-stage fine-tuning framework that achieves 80.46% F1 on Llama-3.1-8B while reducing human verification time by 41%.

Controlling Creativity in Concept Space: A Framework for Generative Models

Presents Concept-Level Temperature (Concept-τ), a framework that maps underspecified prompt elements into a structured concept graph and allocates novelty across discrete surprise levels using convex transforms to achieve predictable, monotonic, and utility-preserving creativity modulation.

Decoupled Contrastive Decoding via Expert-Aligned Drafting

Addressing the severe autoregressive latency overhead incurred by the amateur model pass in Contrastive Decoding (CD), this paper diagnoses why lightweight drafters fail to absorb contrastive residuals, and introduces Decoupled Contrastive Decoding (DCD) to remove the amateur model from the serial drafting path, achieving 1.65x to 1.95x speedups while provably preserving vanilla CD's output distribution.

Demographic Prompting at Scale: When More Attributes Hurt LLM–Human Agreement

Through an exhaustive combinatorial evaluation across five subjective language tasks and five open-source LLMs, this work demonstrates that full-attribute demographic prompts never yield optimal model–human alignment and frequently degrade performance below unprompted baselines, introducing a dataset-level diagnostic framework (SHAP importance, LinearSVC learnability, and Fisher directional coherence) alongside response-token neuron probing to explain alignment dynamics.

DiffuTester: Accelerating Unit Test Generation for Diffusion LLMs via Mining Structural Pattern

DiffuTester accelerates unit test generation in diffusion LLMs by 2x–3x without compromising test coverage, dynamically extracting shared syntactic skeletons across concurrent test cases via line-granularity AST merging while isolating literal values to protect test input diversity.

Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs

PsPLUG frames personalization as a conditional distributional residual against style-guided reference distributions, injecting a compact 3-token continuous prefix tuned via Bradley-Terry preference optimization to resolve "personalization collapse" and enable continuous inference-time strength control.

Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding

Addressing the quality-cost trade-off between sequential autoregressive drafting and parallel block drafting in speculative decoding, Domino decouples intra-block causal modeling from draft execution by combining a single-pass parallel backbone with a lightweight low-rank logit correction head, yielding superior acceptance lengths with minimal drafting overhead.

Entropic properties of languages in phonetic versus orthographic alphabets

Using the DoReCo natural speech corpus spanning 42 typologically diverse languages across 25 families, this paper computes continuous Shannon entropy trajectories across word, subword (BPE), and character/phone granularities, demonstrating a systematic modality convergence between orthographic and phonetic encodings with the smallest gap observed at the subword level.

Evo: Autoregressive–Diffusion Large Language Models with Evolving Balance

Evo establishes a mathematical duality between autoregressive and diffusion language generation under a continuous probability flow, using a learned token-wise semantic progression variable and a unified time-conditioned Transformer to adaptively balance coarse global planning with fine-grained local refinement at near-autoregressive inference speed.

Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM

The paper uncovers a fundamental trade-off where expert persona prompts boost alignment-dependent generation and safety refusal while systematically degrading pretrained knowledge retrieval, and proposes PRISM, a fully bootstrapped framework that distills persona-conditioned behaviors into a binary-gated LoRA adapter to achieve multi-task mastery without external data.

From AI Reviewers to Evidence Assistants: Quantifying the Shift in LLM Scientific Peer Review

This paper quantifies the responsibility boundary of LLM peer review across artifact usability, score calibration, panel breadth, and grounded synthesis, demonstrating that AI should serve as an inspectable evidence assistant rather than an autonomous reviewer, and operationalizes this boundary in Review Copilot to substantially boost review actionability and factual grounding under human control.

H²Table: Hierarchical Hypergraph-Enhanced Large Language Models for Complex Table Reasoning

To overcome topological loss and context explosion caused by 1D text serialization and naive flat hyperedge grouping on nested tables, H²Table models complex tables as hierarchical nested hypergraphs, processes them via a four-stage hierarchy-aware encoder, and distills structural embeddings into soft prompts using learnable queries, achieving a 22.88% relative improvement on deeply nested tables.

HiTeC: Hierarchical Contrastive Learning on Text-Attributed Hypergraph with Semantic-Aware Augmentation

HiTeC addresses three core limitations of self-supervised learning on text-attributed hypergraphs—graph-agnostic text encoding, noisy random augmentations, and absence of long-range dependency modeling—via a two-stage decoupled hierarchical contrastive learning framework combining structure-aware text pretraining, semantic-guided hyperedge dropping, and an s-walk subgraph contrastive objective.

Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space

Addressing the exploration-exploitation dilemma in masked diffusion language models where heuristic decoders either collapse into narrow trajectories or explore blindly, this paper proposes Order-Token Search, a decoding algorithm that jointly explores generation order and token space using a stable incremental denoising action likelihood estimator to achieve massive gains across mathematical reasoning and code synthesis without post-training.

InFi-Check: Interpretable and Fine-Grained Fact-Checking of LLMs on Grounding Documents

InFi-Check introduces a controlled synthesis pipeline and a structured diagnostic framework that unifies sentence-level evidence extraction, 6 fine-grained error categories, natural language justifications, and direct corrections, training the InFi-Checker model to surpass frontier LLMs by 27.65% in balanced accuracy.

Jais 2: A Family of Arabic-Centric Open Large Language Models

Jais 2 introduces open-weight, bilingual Arabic-centric foundation models in 8B and 70B parameter scales (the largest trained from scratch), utilizing an expanded 8× FFN ratio, \(\text{ReLU}^2\) activations, a custom 150K BPE vocabulary, and a comprehensive culturally grounded post-training pipeline to set new state-of-the-art results across OALL2, AraGen, and native cultural evaluations.

LDC: Learning to Generate Research Ideas with Dynamic Control

To resolve the inherent trade-offs among novelty, feasibility, and effectiveness in automated research ideation, LDC introduces a two-stage training framework combining supervised fine-tuning with multi-dimensional controllable reinforcement learning, steered at inference time by a sentence-level RNN decoder that dynamically adapts dimension weights according to context.

LLMs Regret Before They Say It: Early Detection and Compositional Architecture of Regret in Hidden States

The paper investigates the internal neural encoding of regret expressions in LLMs under misinformation correction, discovers that regret signals emerge in hidden states several tokens before the word "regret" is generated, and introduces the Regret Dominance Score (RDS) and Group Impact Coefficient (GIC) to uncover a compositional neuron architecture whose targeted suppression eliminates regret expressions.

Micro Language Models Enable Instant Responses

University of Washington and Meta AI introduce micro language models (µLMs, 8.8M–29.5M parameters) that run on-device with 45 ms TTFT to generate and immediately commit the first 4–8 words of a response, masking multi-second cloud delays while prompting the cloud LLM as an asynchronous continuator with three graceful error-recovery mechanisms.

MiST: Mid-Training LLMs for Cybersecurity

MiST reframes cybersecurity LLM adaptation as a deliberate corpus composition problem rather than a token-scaling challenge, transforming a compact, expert-curated seed corpus into structured synthetic mid-training data that substantially outperforms raw domain continual pre-training while providing a markedly superior initialization for downstream task-specific SFT and verifiable RL.

Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space

Addressing the dilemma where heuristic remasking in diffusion language models gets trapped in a single trajectory or explores blindly, this paper proposes Order-Token Search to jointly search over generation orders and token assignments, utilizing an incremental action likelihood estimator for stable pruning to substantially outperform existing decoding baselines and post-training methods.

Not What, But How: A Communicative Audit of LLM Response Framing

Addressing the limitation of conventional cultural benchmarks that focus exclusively on factual correctness while ignoring framing style, this paper proposes FRANZ, a communicative audit framework spanning four sociolinguistic dimensions—cultural positioning, generalizing language, anthropomorphic cues, and Gricean maxims—and demonstrates across SQUARE (a 376k-query corpus from 57 subreddits across 7 countries) that insider positioning and affective anthropomorphism are tightly coupled with pronounced cross-model and cross-national divergence.

PsyAgent: Constructing Human-like Agents Based on Psychological Modeling and Contextual Interaction

PsyAgent introduces a schema-first framework that couples a Big Five Individual Structure (IS) prior with Multi-Scenario Contexting (MSC) frames across eight arenas, utilizing parameter-efficient SFT and DPO to deliver stable trait fidelity and norm-aware contextual behavior in compact language models.

RATF: Robust Adaptive Tri-Encoder Framework for Fake News Detection against Style-Confusion Attacks from LLMs

To resolve the fundamental tension where fake news detectors either fall prey to LLM-driven style-laundering attacks or sacrifice clean accuracy by enforcing rigid style-invariance, RATF decouples news into objective content and style descriptions via LLMs, computes an algorithmic rewritten score via conditional probability curvature, and adaptively calibrates tri-encoder representations through conditioned cross-attention and learnable parametric gating.

Reading Time Modeling with Pixel-Based Language Models

This paper introduces tokenizer-free pixel-based language models (PLMs) into computational psycholinguistics for the first time, formulating a novel Visual Word Prediction Difficulty (VWPD) metric with Gaussian-blurred future context to simulate human parafoveal vision, achieving superior and complementary predictive power over conventional token-based surprisal across diverse multilingual eye-tracking corpora.

Response-Aware User Memory Selection for LLM Personalization

Addressing the flaw that conventional inference-time personalization relies solely on surface similarity between queries and user memories, this paper proposes RUMS, an information-theoretic framework that selects memory subsets by maximizing mutual information with model output distributions, and distills this utility signal into a lightweight DeBERTa classifier that matches or exceeds GPT-4 prompting while slashing token costs by up to 95%.

Reviewing the Reviewer: LLM-Assisted Reviewer Feedback Generation for Guideline Compliance

To combat rampant lazy thinking and vague critiques in academic peer review, this paper introduces the first guideline-aligned feedback generation framework combining neurosymbolic issue detection with an evolutionary reranking algorithm using verifiable composite rewards, reducing review guideline violations by up to 92.4% in a controlled rewriting study.

Select, Label, Evaluate: Active Testing in NLP

This paper introduces the first formal Active Testing framework in NLP, presenting an unsupervised multi-head attention disagreement sampling strategy (Agreement), Horvitz-Thompson unbiased estimation, and an adaptive stopping rule that reduces test set human annotation burdens by up to 95% while bounding model evaluation error within 1% of the full test set.

SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation

Addressing the capacity deficit of on-device small language models and the privacy vulnerability of centralized large models, SpecSteer introduces an asymmetric edge-cloud collaborative framework that formulates inference as Bayesian knowledge fusion, using ratio-based verification to eliminate cloud access to private contexts and contrastive steering recovery to maintain user intent, achieving superior personalized quality with a 2.36x speedup.

Survival-Guided Length Control for Efficient Diffusion Language Models

By reformulating length selection in masked diffusion language models as a discrete-time survival analysis process over the [EOS] token, this paper estimates per-instance expected lengths in closed form from a single initial forward pass, accelerating inference by up to 7x with zero additional training and zero loss in task accuracy.

The Curious Case of Factual (Mis)Alignment between LLMs’ Short- and Long-Form Answers

This paper introduces the SLAQ benchmark to evaluate factual consistency in LLMs between isolated short questions and multi-fact long queries, discovering that high raw alignment is paradoxically driven by shared incorrect answers and compounding momentum dynamics, while mechanistic analysis demonstrates that attention head importance rank correlation reliably predicts answer alignment.

The Curious Case of Factual (Mis)Alignment between LLMs’ Short- and Long-Form Answers

This paper investigates the factual consistency breakdown in large language models when answering identical factual queries in isolation versus embedded within complex long-form prompts, introducing the SLAQ benchmark across 16 models to reveal that high raw alignment is largely dominated by shared incorrect answers, while establishing that circuit similarity—specifically attention head rank correlation—can predict behavioral factual alignment with 80% accuracy (ROC-AUC 0.85).

UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

UltraX introduces an adaptive function-calling refinement framework for pre-training corpora that completes the editing function space by integrating line insertion alongside deletion and modification. Operating via dataset-adaptive prompt optimization, hierarchical text-to-operation mapping, and conflict-resolving post-processing, its 0.6B lightweight refiner achieves up to 2% downstream relative gains across five diverse corpora and matches or exceeds 20B-token baselines using only 16B training tokens.

When Calibration Rankings Reverse: Accuracy-Controlled Evaluation for Fair Comparison of LLMs

Reveals that standard global calibration metrics like ECE and Brier Score are severely confounded by task accuracy, introduces the tripartite ACE (Accuracy-Controlled Evaluation) framework with instance, distribution, and candidate alignment, and demonstrates that apparent calibration gains of larger or thinking models frequently reverse once accuracy is controlled.

When i.i.d. Factorization Breaks Language: The Structural Failure of Discrete Diffusion Language Models

Unveils the fundamental mechanism behind discrete diffusion language models (DDLMs) collapsing into marginal distributions and repeating high-frequency tokens due to i.i.d. corruption and factorized sampling conflicting with language's non-exchangeable structure, establishing the Dependency-Coupling Principle (DCP) for stable parallel generation.

When Surprise Is Not Enough: Validation-Aware Rewards for LLM-Guided Scientific Discovery

Addressing the "noisy-TV" failure mode where open-ended LLM discovery agents driven by Bayesian surprise chase noise and statistical artifacts, this paper introduces Reducible Information Gain (REIG), a validation-aware reward grounded in lower confidence bounds, sign stability, and permutation null calibration that decouples empirical reliability from semantic novelty and non-triviality.

X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests

Addressing the severe data scarcity and benchmark contamination risks in competitive programming, X-Coder establishes the first complete SFT-then-RL post-training lifecycle driven entirely by synthetic tasks, solutions, and executable tests, lifting a 14B model by over 40 points on LiveCodeBench.