๐น Video Understanding¶
๐ฌ EMNLP2026 ยท 17 paper notes
๐ Same area in other venues: ๐ง NeurIPS2026 (41) ยท ๐๏ธ ECCV2026 (111) ยท ๐ท CVPR2026 (190) ยท ๐ฌ ICLR2026 (48) ยท ๐งช ICML2026 (17) ยท ๐ค AAAI2026 (27)
๐ฅ Top topics: Compression ร4 ยท Reasoning ร3 ยท Question Answering ร3
- CapRiCorn-1K: A Comprehensive Benchmark for Video Captioning and Subject Referential Consistency Across Temporal Scales
-
Addressing the critical limitations of existing video captioning benchmarksโshort durations, static scenes, and neglect of identity trackingโCapRiCorn-1K introduces a 1,000-video benchmark spanning 15 seconds to 10 minutes with rich scene transitions and clothing variations, proposing a novel Rand-Index-inspired metric to evaluate subject referential consistency alongside fine-grained keypoint coverage across audiovisual and visual-only settings.
- Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding
-
Caption-once, Frames-on-Demand (CFD) introduces a budget-aware edge-cloud agentic framework for long-video understanding that constructs reusable dual-track narrative memories offline (event skeleton plus clip micro-logs) and deploys an online Visual-Need Router to fetch bounded keyframes only when perceptual questions require direct pixel evidence, reducing per-query frame consumption to 5.8 frames on average while achieving competitive accuracy on Video-MME (67.5) and InfiniBench.
- CausalChapter: Improving Long-Form Video Chaptering with Interventional Dependency Modeling
-
To tackle boundary error propagation and cross-segment context fragmentation in scalable segment-then-caption long-form video chaptering, CausalChapter introduces intervention-inspired predictive dependency modeling via Local Dependency Shift (LCDS) to detect smooth boundary transitions and Cross-Segment Support Selection (CSSE) to rerank historical contexts by output KL divergence shift, substantially outperforming prior state-of-the-art systems on AVLecture and VidChapters-7M.
- CoFIE: Coarse-to-Fine Evidence Selection for Efficient Streaming Video Understanding
-
To tackle high front-end vision encoding costs and the limited latency gains of downstream token pruning in streaming video understanding, CoFiE decouples evidence selection into a pre-encoder coarse filter using grayscale histogram novelty and an LLM prefill fine ranker using text-to-vision attention, achieving up to a 2.54ร end-to-end speedup while outperforming the full-frame baseline.
- Emotion Understanding in Streaming Video with Trajectory-Aware Reliability
-
Addressing the vulnerability of isolated prefix predictions and deceptive single-prefix confidence in streaming video, this paper proposes TRACE, a trajectory-aware reliability framework that continuously tracks emotion belief trajectories from lightweight audio streams, evaluates decision risk via a two-head reliability model, and selectively triggers multimodal contextual reinterpretation, matching full-context performance while reducing reinterpretation calls to 55.58%.
- HIPPO: Accelerating Video Large Language Model Inference via Holistic-aware Parallel Speculative Decoding
-
Targeting the visual token redundancy and draft latency bottlenecks in video-LLM inference, HIPPO integrates a multi-dimensional semantic-aware token preservation strategy with an overlapping video parallel speculative decoding algorithm, achieving up to 3.51ร lossless inference speedup across diverse video benchmarks.
- Long-to-Short Video Evidence Reasoning for Grounded Question Answering
-
Addressing temporal evidence duration imbalance and the optimization conflict between strict boundary overlap and answer reasoning in grounded video QA (GQA), this paper introduces LOVER, an RL post-training framework combining a long-to-short evidence curriculum, an evidence-spotting IoP reward, and adaptive timestamp rendering to achieve new state-of-the-art results among open-source models on NExT-GQA and ReXTime.
- LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video
-
To tackle the challenge of sparse, scattered spatial cues being diluted by redundant visual tokens in long videos, this paper proposes LongSpaceโa unified framework combining 3D Spatial Structure Perception (SSP) and Hierarchical KV Memory (HKM)โalongside LongSpace-Bench, a benchmark spanning seconds to hours of indoor room tours, setting strong state-of-the-art results in long-horizon spatial recall.
- MemDreamer: Decoupling Perception and Reasoning for Long Video Understanding via Agentic Hierarchical Graph Memory
-
MemDreamer decouples long-video understanding into streaming construction of a pure-text Hierarchical Graph Memory and tool-augmented agentic retrieval via an Observation-Reason-Action loop, establishing new SOTA across four benchmarks with a 40โ124ร smaller reasoning context.
- MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answering
-
Addressing the challenge that key evidence in long videos is sparse, brief, and temporally scattered while isolated frames lack coherent semantics, MemoryCard segments videos into topic-coherent sessions via VLM self-reading, renders event gists, speech transcripts, and representative frames into self-contained "Memory Cards", and performs question-conditioned retrieval with adaptive resolution allocation, boosting QA accuracy by up to 21.8% under comparable visual token budgets.
- Reward-Decomposed Reinforcement Learning for Immersive Video Role-Playing
-
EBM-RL decouples video role-playing generation into an explicit See-Think-Speak pipeline and optimizes each stage via four complementary rewards under GRPO, eliminating "atmosphere-blind" responses and achieving situational persona consistency.
- Self-Supervised On-Policy Distillation for Omni-Modal Video Captioning
-
SODA introduces a self-supervised on-policy distillation framework for omni-modal video captioning that exploits intrinsic task and input-budget asymmetries, auditing low-budget student drafts via dual-stream QA and distilling high-budget teacher revisions back into the student through token-level KL divergence without external teachers or human annotations.
- Streaming Atomic Action Segmentation in Egocentric Videos via Iterative Boundary Refinement
-
This paper proposes CoTT (Chains of Temporal Transitions), an annotation-free framework that extracts dense atomic action pseudo-labels from unlabeled egocentric videos via decoupled LVLM transition captioning, language-based grouping, and visual boundary refinement, and distills them into a lightweight causal online model (CoTT-Lite) whose predictions iteratively bootstrap subsequent training rounds to surpass fully supervised online action localization baselines.
- TempCore: Are Video QA Benchmarks Temporally Grounded?
-
The paper introduces Frame Selection Sensitivity (FSS) based on the contrast between most and least relevant frames and the Language Independence Score (LIS), auditing six popular Video QA benchmarks to reveal that merely 5.5%โ31% of samples genuinely demand temporal frame selection, and constructs the compact, temporally grounded benchmark subset TEMPCORE without manual re-annotation.
- ToM-Video: A Theory-of-Mind Video Benchmark with Spatio-Temporal Grounded Reasoning
-
This paper introduces ToM-Video, a large-scale spatio-temporal grounded Theory-of-Mind benchmark comprising 1,487 real-world narrative videos and 41,206 multiple-choice questions, and develops INSPIRE, a three-stage curriculum post-training framework that progressively aligns format compliance, spatio-temporal evidence grounding, and causal mental-state reasoning, substantially narrowing the gap between open-source models, proprietary frontier systems, and human experts.
- VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration
-
Addressing the superficial shortcuts of multiple-choice QA and the sparsity of human caption matching, this paper introduces VidOmni-Bench, a sentence-wise event verification benchmark spanning 5 spatio-temporal complexity types and durations from 4 seconds to 90 minutes that uses human-verified Video-LLM hallucinations as hard negatives, systematically diagnosing temporal reasoning bottlenecks and self-preference bias.
- VISA: Video Skeleton-Aware Efficient Frame Selection for Long Video Understanding
-
VISA utilizes sparse native I-frames from video Group-of-Pictures (GOP) structures to build a query-agnostic temporal skeleton, aggregates hierarchical query relevance, allocates frame budgets via dynamic programming, and refines candidates across neighboring GOPs, reducing end-to-end latency by 32.9%โ53.6% while outperforming prior selection baselines.