๐ป Code Intelligence¶
๐ง NeurIPS2026 ยท 44 paper notes
๐ Same area in other venues: ๐๏ธ ECCV2026 (2) ยท ๐ท CVPR2026 (1) ยท ๐ฌ ICLR2026 (59) ยท ๐ฌ ACL2026 (49) ยท ๐งช ICML2026 (22) ยท ๐ค AAAI2026 (10)
๐ฅ Top topics: Code Intelligence ร12 ยท LLM ร7 ยท Agents ร5 ยท Diffusion Models ร2 ยท Reasoning ร2
- A Theoretical Analysis of Test-Driven Code Generation
-
The paper provides a unifying probabilistic and decision-theoretic foundation for test-driven code generation, proving that soft functional similarity estimators strictly dominate hard equivalence in signal-to-noise ratio and formalizing iterative backprompting as in-context Thompson sampling with an irreducible regret bound.
- Build-and-Find: An Effort-Aware Protocol for Evaluating Agent-Managed Codebases
-
Build-and-Find establishes an effort-aware evaluation layer between behavioral correctness and downstream modification by treating generated codebases as communication artifacts and measuring downstream agents' intent recovery accuracy and inspection effort under strict stability gates.
- Code2World: A GUI World Model via Renderable Code Generation
-
Code2World shifts GUI next-state prediction from raw pixels or abstract text to renderable HTML code generation, aligning layout fidelity and action logic via dual-reward Render-Aware Reinforcement Learning (RARL) to serve as a high-fidelity virtual sandbox that enhances downstream agent navigation.
- CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion
-
Targeting the "safety generalization lag" where natural language safety alignment fails to generalize to structured programming, this paper proposes CodeMimicry, a fully automated black-box jailbreak framework that induces harmful code completion via object-oriented encapsulation and iterative refinement, achieving a 96.25% attack success rate with only 1.51 queries on average across 8 commercial LLMs.
- CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
-
CodeScaler distills high-quality execution feedback into a code reward model, uses strict code extraction and reward shaping to support reinforcement learning and candidate reranking without online execution, and exceeds RLVR by 1.55/4.23 percentage points in average Avg@8 across four benchmarks for 8B/14B policies trained on DeepCoder.
- Constrained Code Generation with Discrete Diffusion
-
Addressing the shortcomings of autoregressive post-hoc repair and syntax-only constrained decoding, this paper introduces CDC, a training-free neurosymbolic framework that integrates soft surrogate execution signals and static program analysis directly into the discrete diffusion reverse denoising trajectory to enforce functional correctness and security constraints with surgical locality.
- CORVUS: Context Optimization and Reduction Via Underlying Synchronization for LLM Coding Agents
-
Addressing the inefficiencies of traditional append-only trajectory architectures that cause duplicate file reads and stale snapshot errors in LLM coding agents, CORVUS decouples file-reading actions from observations via a synchronized registry that injects refreshed repository states before each reasoning cycle, cutting 9โ50% input tokens and up to 37% reasoning cycles while maintaining task pass rates.
- Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents
-
CRR restores a small number of decision states during software-agent training and scores realized actions using the terminal-return difference between the original trajectory and an alternative-action continuation, raising SWE-bench Verified pass@1 from extended GRPO's 36.7% to 41.7% within the same 40.125-hour training window on an eight-H100 node, without eliminating fork execution costs.
- CRANE: Constrained Reasoning Injection for Code Agents via Nullspace Editing
-
CRANE introduces a training-free directional parameter-editing framework that injects planning and error-recovery behaviors from Thinking models into Instruct code agents via magnitude thresholding, Conservative Taylor Gating, and Graduated Sigmoidal Projection, boosting task success across Roo-Eval, SWE-bench-Verified, and Terminal-Bench v2 while strictly maintaining compact, tool-disciplined execution.
- CTE-Bench: Counterfactual Trace Evaluation for Stateful Software Simulators
-
CTE-Bench executes software with and without an intervention to evaluate sustained response prediction on fixed future calls: effect-step accuracy reaches 54.3%โ61.5% with correct earlier answers, falls to 24.8%โ33.2% in free rollout, and whole-trace exact match peaks at just 1.2%.
- CUDABeaver: Benchmarking LLM-Based Automated CUDA Debugging
-
Introduces CUDABeaver, the first benchmark dedicated to automated CUDA debugging starting from real LLM failing workspaces, uncovering the widespread "repair by degeneration" failure mode and formalizing multi-dimensional protocol-conditional evaluation metrics.
- CUDABench: Benchmarking LLMs for Text-to-CUDA Generation
-
Presents CUDABench, the first benchmark spanning six scientific computing domains across multi-scale inputs and graded prompt difficulties for direct text-to-CUDA generation, introducing a roofline-based hardware-independent metric (CUDABench-Score) and an automated generative verification pipeline that reveals major bottlenecks in semantic correctness and hardware efficiency.
- DCVD: Dual-Channel Cross-Modal Fusion for Joint Vulnerability Detection and Localization
-
Addressing the reliance on single information sources and the absence of explicit line-level supervision in conventional vulnerability detectors, DCVD builds a dual-channel architecture fusing AST/CFG control dependency graphs with LLM semantic explanations via contrastive alignment and bidirectional cross-attention, synergistically optimized with explicit dual-granularity supervision.
- Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests
-
To tackle test-gaming where coding agents achieve deceptively high scores by exploiting accessible test cases, this paper proposes CapCode, which injects uniform random values into tests to enforce a theoretical performance cap for statistical detection, and CapReward, a non-monotonic reward function that penalizes exceeding the cap during reinforcement learning to suppress reward hacking.
- Execution Guided Line-by-Line Code Generation
-
At inference time, EG-CFG previews and executes several short continuations of a code prefix, injects runtime traces into the prompt, and generates code token by token with dual-distribution classifier-free guidance while refreshing feedback at line boundaries; with DeepSeek-V3-0324, the paper reports 96.6% / 99.4% / 69.9% accuracy on MBPP / HumanEval / DS-1000, respectively, but requires executable provided tests and additional search computation.
- FORALL-LEAN-AGENT for Auditable Reasoning in Formal Mathematics and Software Verification
-
Addressing the critical vulnerability that compiler pass alone does not guarantee intended statement validity or rule compliance in autonomous Lean proof generation, this paper presents Forall-Lean-Agentโa frontend-agnostic execution and acceptance protocol combining runtime blindness, statement-only retrieval, stateless fresh review, and multi-tier axiom/independent kernel auditing.
- Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models
-
Addressing the trade-off between slow one-token-per-step static decoding and quality-degrading parallel decoding in Diffusion Large Language Models (DLLMs), FreeDave introduces a training-free and model-free self-verifying parallel decoding algorithm that provably reproduces the exact static decoding trajectory with minimal forward passes, accelerating inference by up to 2.83ร across math and code tasks without accuracy loss.
- From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
-
P2T (Patches-to-Trajectories) distills developer reference patches as privileged information into leakage-free latent process graphs and uses a receding-horizon bi-objective selection rule to curate short, grounded process-supervision trajectories that boost SWE-bench Pass@1 while reducing inference costs.
- Frontier Coding Agents Use Metaprogramming to Adapt to Unfamiliar Programming Languages
-
Evaluates frontier coding agents on esoteric programming languages within interactive execution environments, revealing that top-tier agents adapt not by direct authoring in the target language, but by autonomously inventing host-language generator compilers and verifying them locally.
- FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale
-
FrontierSmith transforms closed-ended competitive programming problems into open-ended optimization tasks lacking polynomial optimality certificates via three orthogonal formulation mutations, filters them via a two-stage idea divergence metric, and generates continuous reward environments using cross-validating test-case and verifier agents to achieve reinforcement learning gains matching or surpassing human-curated datasets on Qwen3.5.
- fxBench: Evaluating and Understanding Formula Suggestions in Spreadsheets
-
Addressing the lack of real-world benchmarks and capability decomposition for in-line formula suggestions in spreadsheets, this paper presents fxBenchโa leakage-controlled benchmark of 503 verified tasksโproposes the compact Minisheet representation, and reveals that inferring intent rather than syntax generation forms the primary performance bottleneck.
- LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models
-
Introduces LibEvoBench and the composite SEUS metric, revealing that frontier code generation LLMs suffer from severe "version-oblivious" degradation and anachronistic errors on evolving APIs while failing to leverage explicit version constraints.
- MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization
-
MM-IssueLoc addresses the disconnect between real-world screenshot-heavy issue reports and existing text-only or end-to-end patch benchmarks by establishing a controlled suite of 652 multilingual issue-PR instances and 1,050 images, decoupling visual format from structured content to reveal a substantial capability gap across LLM agents and dense retrievers.
- MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents
-
Introduces MOSAIC-Bench comprising 199 three-stage innocuous engineering ticket attack chains with deterministic exploit oracles, revealing that production coding agents achieve 53โ86% exploit ASR while bypassing both refusal and hardening safety reflexes.
- Murphy: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation
-
Murphy extends single-turn GRPO to feedback-conditioned rollout trees and applies retrospective credit assignment (MaRS/MeRS) combined with variance-aware tree pruning, achieving up to a 6.20 percentage point pass@1 gain over prior execution-feedback methods across multi-turn self-correction benchmarks.
- P\(^{3}\): Joint Program-and-Proof Planning for Verified Code Generation
-
Addressing the structural mismatch in decoupled "program-then-proof" pipelines, \(P^3\) introduces joint program-and-proof planning that co-derives computational decomposition and bridging predicates prior to elaboration, achieving superior solve rates and efficiency across Lean 4 benchmarks including the new repository-grounded Lean4Commit0.
- PBT-Bench: Benchmarking AI Agents on Property-Based Testing
-
Addressing the limitation that existing code benchmarks only measure concrete test reproduction or patch generation rather than abstract invariant verification, this paper introduces PBT-Bench, the first property-based testing benchmark spanning 100 problems across 40 real Python libraries with 365 curated semantic bugs and a dual-container \(F \to P\) harness, revealing critical gaps in frontier LLMs' ability to translate documentation into targeted input generators.
- Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?
-
To address the pervasive illusion where frontier LLMs pass debugging unit tests by wholesale code regeneration rather than targeted fault fixing, this paper proposes PDB, a dataset-agnostic benchmarking framework with verified atomic bug composition and precision-aware metrics (edit-level precision and bug-level recall).
- Property-Guided LLM Program Synthesis for Planning
-
This paper proposes a property-guided LLM program synthesis framework to synthesize Python heuristic functions for classical planning domains, verifying a formal "direct" property and feeding back concrete counterexample states upon violation to drive iterative repair; compared to sample-and-select baselines, it generates 7.4x fewer candidate programs, slashes evaluation computation by over three orders of magnitude, and solves more out-of-distribution tasks via pure hill climbing without combinatorial search.
- Remember Your Trace: Memory-Guided Long-Horizon Agentic Framework for Consistent and Hierarchical Repository-Level Code Documentation
-
MemDocAgent combines a traversal order constrained by dependencies and directory hierarchy with a readable, writable, and verifiable RepoMemory to let one agent continuously produce component-, module-, and repository-level documentation; across 20 Python repositories, its Qwen3-Coder configuration achieves 0.979 completeness, 0.916 truthfulness, and 0.690 helpfulness, but its coverage-preserving fallback does not guarantee that every document is trustworthy.
- SaaSBench: Exploring the Boundaries of Coding Agents in Long-Horizon Enterprise SaaS Engineering
-
Addressing the limitation that existing coding agent benchmarks focus on toy or single-stack projects without real enterprise constraints, this paper introduces SaaSBench, the first benchmark spanning 30 market-grounded enterprise SaaS systems built and deployed from scratch, revealing through a dependency-aware DAG hybrid evaluation that system integration rather than isolated business logic is the primary failure bottleneck.
- Sandboxed Coding Agents are Competitive Omni-modal Task Solvers
-
This paper discovers that sandboxed text+image coding agents can match or surpass native omnimodal models across audio-video benchmarks by dynamically running terminal tools (ffmpeg, Whisper, OCR) rather than ingesting dense media streams, and introduces the Code-X open-source post-training recipe alongside the TerminalBench-O processing benchmark.
- Self-Evolving Coding Rules for AI Coding Agents
-
RuleEvolve is an automated closed-loop framework that evolves long-document coding rules for AI coding agents using LLM-powered mutation, execution-based verification, and strategic mutation allocation based on Optimal Computing Budget Allocation (OCBA), cutting generated code length and token inference costs significantly while strictly maintaining functional correctness.
- SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks
-
Introduces SlopCodeBench, an iterative benchmark of 36 problems across 196 checkpoints evaluating coding agents on evolving specifications, revealing severe structural erosion and verbosity accumulation across trajectories.
- Solver-Aware Decompositions for Programming-by-Example: When Dividing Requires Knowing How to Conquer
-
Addressing the "accuracy paradox" in decomposition-based Programming-by-Example (PBE) where high ground-truth alignment fails to yield synthesis success, Solver-Aware Decompositions (SAD) optimizes the decomposer online with self-critical policy gradients rewarded by a frozen neural synthesizer's cross-entropy loss to propose solver-tractable subgoals.
- StaminaBench: Stress-Testing Coding Agents over 100 Interaction Turns
-
StaminaBench introduces a long-horizon benchmark evaluating coding agent stamina over 100 consecutive interaction turns via procedurally generated REST API schema modifications and black-box HTTP tests, finding that all tested open-source models fail within 6 turns without test feedback.
- StepCAD: Mesh-to-CAD Code Generation via LLM Policy and Geometry-Guided Search
-
Addressing error accumulation in single-pass mesh-to-CAD code generation, StepCAD introduces a generative optimization framework combining a state-conditioned CAD policy with IoU-guided tree search over executable program edits, delivering up to an 87.2% relative IoU improvement on complex geometries.
- SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades
-
Presents SWE-Chain, the first benchmark evaluating coding agents on continuous chained release-level upgrades across real Python packages, using DecompSynth to ground specifications in release notes and code diffs while uncovering critical agent bottlenecks in preventing regressions across version transitions.
- SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution
-
SWE Atlas introduces an expert-crafted benchmark suite of 284 tasks spanning Codebase Q&A, Test Writing, and Refactoring, pairing programmatic execution checks with atomic rubric grading to evaluate software engineering rigor and code maintainability beyond superficial functional resolution.
- TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steering
-
Addressing agent drift in long-horizon coding tasks where agents repeatedly re-derive known facts (overthinking) or fire redundant tool calls (overacting), TACT projects residual stream activations at the reasoning-action boundary onto orthogonalized contrastive drift axes and adaptively pulls them back into the calibrated region at test time, boosting resolve rate by up to +5.8 pp on SWE-bench Verified while reducing execution steps by up to 26%.
- The Complexity Kink: A Prompt-Side Structural Complexity Index for Code-Generation Reliability
-
To address the measurement bias in cyclomatic complexity caused by truncated failure code, this paper proposes an input-only, six-dimension prompt structural complexity index paired with Hansen's threshold regression, uncovering how task-type and construction-frame composition drive apparent reliability jumps.
- Veri-Sure: Multi-Agent RTL Code Generation with Temporal Tracing, Slicing and Formal Verification
-
Addressing sparse simulation feedback, whole-file regression, and cross-turn semantic drift in LLM-based RTL synthesis, Veri-Sure establishes a multi-agent closed loop featuring structured design contracts, waveform temporal tracing, static dependency slicing for localized patching, and dual formal verification, achieving a state-of-the-art 93.30% functional Pass@1 on VerilogEval-v2-EXT.
- Verifiers in the Loop: Decoding Time Verification for Code Translation
-
Introduces Decoding Time Verification (DTV), an inference framework that interleaves code generation with deterministic verifier checks at structural boundaries, using structure-aware rollback and diagnostic feedback augmentation to significantly improve compilation pass rates under matched token budgets.
- VeriPy: Source-Preserving Verification and Compatibility Checking for Python Components
-
VeriPy enables functional verification and scoped backward compatibility checking for Python components by combining non-intrusive
#@comment annotations and typed admission gates with dual Dafny/Lean AST encoders and sequential relational product programs.