⚖️ Alignment & RLHF

🎞️ ECCV2026 · 6 paper notes

📌 Same area in other venues: 📷 CVPR2026 (12) · 🔬 ICLR2026 (102) · 💬 ACL2026 (38) · 🧪 ICML2026 (37) · 🤖 AAAI2026 (17) · 🧠 NeurIPS2025 (36)

🔥 Top topics: Alignment/RLHF ×2

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis

Omni-RRM synthesizes preference data with five-criterion justifications through dual-teacher consensus, then uses SFT→GRPO to learn pairwise response discrimination across images, video, and audio, raising the 7B backbone's five-benchmark average accuracy from 60.2% to 70.4% and improving response selection without updating the generator's parameters.

Don't Let the Video Speak: Audio-Contrastive Preference Optimization for Audio-visual Language Models

For the cross-modal hallucination in which audio-visual language models (AVLMs) invent sounds from what they see, this paper proposes ACPO: preference pairs built along two orthogonal axes — an output-contrastive axis that penalizes visual captions offered as answers to audio-focused prompts under an audio-swapped input, and an input-contrastive axis that requires an answer to become untenable once its supporting audio track is replaced — which, while fine-tuning only the audio projection layer, lifts AVHBench audio-hallucination F1 from 79.0 to 80.4 and unimodal audio-captioning CIDEr from 33.0 to 43.6.

D2PO: Optimizing Diffusion Samplers via Dynamic Preference

D2PO reformulates diffusion sampler parameter optimization from a rigid student-teacher regression task into a preference alignment problem using an energy-based surrogate and dynamic self-refining teachers, markedly enhancing perceptual fidelity under ultra-low NFE budgets.

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

HumanTracker introduces a 150-hour optical mocap benchmark across four motion families and HumanScore, a temporal Transformer trajectory reward model trained on 12K human preference pairs, achieving 92.91% preference alignment and revealing contact failures missed by kinematic metrics.

Reward Modeling for Computer-Using Agent from Video Execution

Addressing the reliance of computer-use agent (CUA) evaluation on handcrafted rules or internal reasoning traces, this paper introduces ExeVRM, a model-agnostic reward model evaluated directly on interface execution videos, enabled by the 53k-scale ExeVR-53k dataset, adversarial instruction translation for hard negative synthesis, and spatiotemporal token pruning (STP+TTP) that keeps 720p long-horizon evaluation tractable while outperforming GPT-5.2 and Gemini-3 Pro.

VERTIGO: Visual Preference Optimization for Cinematic Camera Generation

VERTIGO introduces the first visual preference optimization post-training framework for camera trajectory generators by rendering 3D camera paths into 2D previews via a real-time graphics engine (Unity), scoring them using a cinematically adapted VLM via inverse captioning consistency, and applying DPO to align camera motions with shot composition, reducing target off-screen rates from 38.7% to 0.8%.