ECCV2026 Video Understanding AI paper notes paper summaries Object Tracking Compression Human Pose Anomaly Detection Few-/Zero-Shot Learning Multimodal/VLM

📹 Video Understanding¶

🎞️ ECCV2026 · 110 paper notes

📌 Same area in other venues: 📷 CVPR2026 (187) · 🔬 ICLR2026 (48) · 🧪 ICML2026 (17) · 🤖 AAAI2026 (27) · 🧠 NeurIPS2025 (39) · 📹 ICCV2025 (56)

🔥 Top topics: Object Tracking ×18 · Compression ×7 · Human Pose ×7 · Anomaly Detection ×6 · Few-/Zero-Shot Learning ×4

EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use: EventMemAgent compresses video history into hierarchical event-centric memory and uses reinforcement learning to teach Qwen3-VL-8B when to retrieve, perform OCR, or detect objects, achieving an overall score of 60.75% on OVO-Bench with at most 32 frames in short-term memory.
History-Aware Transformation of ReID Features for Multiple Object Tracking: HATReID-MOT treats established tracks as online pseudo-classes, constructs a video-specific ReID subspace using temporally weighted Fisher linear discriminant analysis, and fuses original similarities, raising the default appearance baseline's SportsMOT test HOTA from 67.3 to 78.1 without additional network training.
MegaFlow: Zero-Shot Large Displacement Optical Flow: MegaFlow builds cross-frame features from pretrained visual geometry priors, locates large motions through global matching, and refines them with local and temporal evidence, achieving 1.83 EPE in four-frame zero-shot Sintel Final evaluation while supporting transfer to point tracking without tracking supervision.
TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding: TAR inserts intermediate anchors verifiable against ground-truth intervals into video reasoning, constrains reinforcement learning with progressive refinement and an anchor-count penalty, and bootstraps cold-start data with a same-scale model, reaching 61.1 mIoU on Charades-STA with a 7B model.
Bounding-Box Trajectories Matter for Video Anomaly Detection: TrajVAD promotes bounding-box trajectories already produced by detection and tracking to the primary anomaly signal, learning normal motion with a class-aware normalizing flow; its trajectory-only variant reaches 87.7 AP on ShanghaiTech, while reliability-gated pose fusion reaches 88.6 AUROC and 90.9 AP, although pose does not help on every dataset.
CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Management: CurveStream detects semantic changes through the turning behavior of visual feature trajectories and uses online dual thresholds to retain frames at high resolution, retain them at low resolution, or discard them, improving Qwen2.5-VL-7B from 73.31% / 59.90% to 84.00% / 73.48% on StreamingBench / OVOBench without training.
EgoExo-Con: Exploring View-Invariant Video Temporal Understanding: This paper introduces the EgoExo-Con benchmark (1,148 pairs of synchronized ego-exo videos + 2,269 human-refined temporal queries) to systematically evaluate the cross-view consistency of Video-LLMs in temporal understanding for the first time, revealing that existing models' cross-view consistency barely reaches half of their single-view performance. Furthermore, it proposes the View-GRPO reinforcement learning framework, which leverages dual reasoning rewards of semantic alignment and structural consistency to significantly enhance the robustness and consistency of cross-view temporal understanding.
Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding: Em-Garde reformulates the complex "whether to answer" reasoning performed at every frame into a one-time visual proposal generation during query time and lightweight in-stream matching. It achieves an average F1 of 30.99 on OVO-Bench, 38.0% proactive response accuracy on StreamingBench, and maintains constant-length streaming computation at approximately 13 fps.
OmniScript: Towards Audio-Visual Script Generation for Long-Form Cinematic Video: OmniScript transcribes cinematic videos into temporally grounded hierarchical scripts, combining character-memory-assisted data construction, deep audio-visual fusion, plot reasoning, and event-level segmented rewards to achieve 37.7 event Overall F1 and 69.3 [email protected] with an 8B model on 5-minute videos, without uniformly surpassing proprietary models.
Small Vision-Language Models are Smart Compressors for Long Video Understanding: Tempo uses a 2B small vision-language model to compress video segments into query-conditioned memories for a 4B global language model, allocating tokens by relevance at inference time; with at most 1024 frames and a 4K visual budget, it scores 52.7 on LVBench versus 48.2 for VideoChat-Flash.
VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflection: VDC-Agent uses one multimodal model to generate, evaluate, and reflect on video captions, then applies easy-to-hard DPO to trajectory-derived preference pairs, turning iterative improvements into single-pass generation that achieves 49.08% average accuracy on VDC and 37.4% F1 on DREAM-1K.
IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video: IRIS is the first real-world benchmark for physical parameter estimation that covers multi-body collision phenomena: 240 controlled 4K/60fps recordings across 8 dynamical systems (24 settings, 10 repeated trials each), with ground-truth parameters independently measured together with uncertainty, plus a five-axis evaluation protocol and the first systematic comparison of four equation-routing strategies. Baseline evaluation not only quantifies accuracy but also diagnoses a gradient-flow bug in the state-of-the-art latent-space pipeline (fixing it cuts pendulum rope-length MAE from 95.07 m to 3.80 m) and exposes catastrophic divergence of a multi-step rollout loss on multi-body systems (worst two-pendulum rope-length MAE: 741.1 m).
Synthetic Visual Genome 2: Extracting Large-scale Spatio-Temporal Scene Graphs from Videos: SVG2 scales automatic annotation of real videos to 636K videos and trains the trajectory-conditioned TraSeR with segmentation alignment and global/local dual resamplers, achieving 79.0% object accuracy and 27.1% attribute recall on SVG2test, although its relation predictions do not consistently surpass GPT-5.
DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding: DART uses one query-conditioned DPP kernel for both keyframe selection and difficulty estimation, activates structured temporal reasoning only for complex events, and achieves 48.93 mIoU on Charades-STA and 39.89 on ActivityNet Captions without task-specific fine-tuning.
Reflect-R1: Evidence-Driven Reflection for Self-Correction in Long Video Understanding: Reflect-R1 uses one vision-language model to retrieve evidence and answer initially, verify under an isolated context, and arbitrate after renewed evidence gathering, learning correction rather than repetition through SD-GRPO and improving accuracy from 59.68% to 62.50% in the LongVideoBench reflection analysis.
STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding: STRIDE uses a masked diffusion activation model initialized from Qwen3-VL-2B to jointly revise response decisions within a sliding window before invoking a frozen Video-LLM, improving mean F1 in ET-Bench online activation evaluation from Baseline-AR's 24.3 to 32.6.
Video-Oasis: Rethinking Evaluation of Video Understanding: Rather than building yet another video benchmark, Video-Oasis proposes a sustainable diagnostic suite that audits 14 existing video understanding benchmarks along three axes — removing visual evidence, destroying temporal structure, and human-in-the-loop verification of annotation ambiguity — finding that 55% of samples are answerable without visual or temporal information; on the resulting 11,033 video-native challenge samples, state-of-the-art models sit only marginally above the 25.6% random-chance baseline.
Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos: Vinci2 reframes "should an egocentric assistant speak up on its own?" as a decision problem over long-horizon context, and delivers both EgoServe — the first benchmark for proactive assistance in continuous egocentric video (3,000+ service instances, 4 temporal memory horizons, 10 subcategories) — and EgoMemo, a training-free memory-augmented agent (multi-scale temporal summaries + an evolving knowledge graph + a visual embedding archive, with three parallel retrieval pathways and VLM caption reconstruction), which lifts overall F1 on EgoServe from 4.7 (GPT-5-mini baseline) to 8.0 while remaining competitive on five existing benchmarks.
Cambrian-P: Pose-Grounded Video Understanding: Cambrian-P adds per-frame camera pose regression to video MLLM training and reconciles geometry learning with question answering through interleaved training and frame sampling jitter, improving VSI-Bench from 69.2 to 73.7 against a matched no-pose baseline; the gains primarily reflect better learned representations rather than inference-time pose-token use.
ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control: ConTrack treats object motion tracking as the priority task constraint and learns robotic hand control with online weight adaptation, a reachable-state reset library, and contact priors, achieving mean progress of 0.899 and contact F1 of 0.784 within 5000 PPO updates per clip, without leading every error metric.
Learning Consistent Temporal Grounding between Related Tasks in Sports Coaching: Exploiting the observation that two tightly related tasks — generating technique feedback and verifying whether a feedback statement is correct — must attend to the same keyframes, this paper enforces cross-pathway attention consistency over selected vision-centric heads of a shared Vid-LLM (with a stop-gradient verification pathway as the reference), improving ExAct / FitnessQA / ExpertAF by +3.0 / +14.1 accuracy and +0.9 BERTScore over supervised fine-tuning without any frame-level annotations during training.
Learning Probabilistic Embeddings for Unsupervised Action Segmentation: PEOT predicts Gaussian frame embeddings with a local temporal graph, trains with multiple sampled optimal-transport pseudo-labels and cross-entropy losses, and uses only the means at test time, improving Desktop Assembly MoF from 59.0 to 71.2 and F1 from 63.7 to 75.8 over a code-reproduced ASOT baseline without leading on every metric of every dataset.
OnPoint: Offline-to-Online Multi-Level Distillation for Point-Supervised Online Temporal Action Localization: OnPoint proposes an offline-to-online multi-level distillation framework. It utilizes an offline teacher model trained with only single-frame point annotations to generate pseudo-segment labels, frame-level Class Activation Sequences (CAS), and window-level action anticipation signals. These are injected into a strictly online student model through three levels of distillation (instance-level, frame-level, and window-level), complemented by actionness-calibrated attention decoding and anchor-level raw point supervision to stabilize training. OnPoint consistently outperforms strong baselines across five datasets (with an average mAP improvement of 4.8% and a maximum of +7.0% on THUMOS), initiating the novel task of Point-supervised Online Temporal Action Localization (POTAL).
QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding: QCA splits a fixed frame budget across temporal segments using two signals — query semantic matching and content deviation — and then, within each segment, anchors on the most query-relevant frame and greedily adds the frames farthest from the selected set; it is fully training-free and plugs into existing Video-LLMs, consistently beating uniform sampling and prior frame-selection methods on four long-video benchmarks across three MLLM backbones (with Qwen3-VL-8B at a 64-frame budget: 66.9 / 69.5 / 75.7 / 51.8 on LongVideoBench / Video-MME / MLVU / LVBench).
ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA: ReQuest distills a lightweight selector from an MLLM's own answers to estimate how much visual evidence each frame contributes, then reuses the prediction entropy of the first inference pass to decide both whether a second thinking round is warranted and how densely frames should be sampled, improving long-video QA accuracy without fine-tuning the underlying MLLM and spending the extra inference only on questions the model is genuinely uncertain about.
Revisiting Weakly-Supervised Video Scene Graph Generation via Pair Affinity Learning: This paper shows that weakly-supervised video scene graph generation (WS-VSGG) inherits the fully-supervised two-stage pipeline without accounting for a fundamental shift: an off-the-shelf detector indiscriminately detects objects irrelevant to any interaction, yet these unmatched pairs are discarded at training time, so the model is trained only on interactive pairs while at inference it must rank over a detection space dominated by non-interactive ones. The authors introduce a learnable pair affinity — RAM refines pseudo-labels via vision-language grounding, PALS multiplies an affinity score into inference-time ranking, and PAM gates spatial and temporal attention by affinity — raising weakly-supervised STTran from 15.39 to 22.24 R@10 on Action Genome.
TETO: Tracking Events with Teacher Observation for Motion Estimation and Frame Interpolation: TETO transfers correspondence knowledge from an RGB tracker to events using about 25 minutes of real paired recordings without manual motion annotations, improves trajectory and flow estimation through object-motion-aware sampling, and uses explicit motion constraints to improve the perceptual quality of diffusion-based frame interpolation.
TransVLM: A Vision-Language Framework and Benchmark for Detecting Any Shot Transitions: TransVLM replaces isolated shot boundaries with complete transition intervals, combining color and optical flow without lengthening the visual-token sequence, mixed-data training, and sliding-window inference to achieve 0.783 segment-level F1 on public STD Benchmark data and 0.938 frame-level F1 on synthetic data.
Triangular Consistency as a Universal Constraint for Learning Optical Flow: Starting from the geometric first principles of compositionality of displacement fields, this paper proposes triangular consistency—a universal optical flow constraint that is independent of architecture, supervision type, and dataset. By enforcing composition consistency loss on the minimal triangle formed by three frames, consistent improvements are achieved across supervised, unsupervised, and self-supervised domain adaptation paradigms.
WristMimic: Full-Body Humanoid Control with Wrist-Guided Manipulation: WristMimic separates human–object interaction retargeting into two supervision regimes — kinematic pose guidance for contact-free body parts and wrists, and fingers shaped only by object trajectory and contact outcomes — and secures wrist placement through reward modulation and phase-specific reset thresholds within a contact window; across 20 sequences each from OMOMO and ParaHome it attains a 91.1% average success rate, versus 43.3% for InterMimic despite the latter's full finger pose supervision.
Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensation: The method compresses action-feature sequences into sparse DDIM latent anchors and allocates anchors by reconstruction error, reaching 68.0% ASFormer accuracy on Breakfast versus 62.8% for GNI at the same temporal anchor budget, while remaining below the 70.7% full-data result.
Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction: This paper proposes TASKER, a generalized algorithm that formulates keyframe extraction as a graph search problem. An MLLM simultaneously evaluates task relevance (what information is missing) and scene dynamics (where major shifts occur) to guide the search direction, paired with a dual-path confidence voting mechanism to decide when to terminate. TASKER outperforms the previous best baselines by 2.0% and 1.8% on EgoSchema and NExT-QA respectively using only about 15% of the total frames, and introduces the matching VG-GUI-Bench benchmark to evaluate the model's ability to learn step-by-step operational steps from video tutorials and transfer them to GUI agent tasks.
ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement: ClearText-Video (CTVid) connects low-level restoration with text-grounded video QA through high-quality, degraded, and restored versions of the same content, using 4,639 videos to show that sharper-looking inputs need not be read more accurately and that in-dataset fine-tuning can improve accuracy while reducing an uncertainty-aware metric.
DnA: Denoising Attention for Visual Tasks: DnA uses softmax/softmin queries over shared keys to retrieve complementary interactions, then combines separate value projections with a learned per-head coefficient, raising ViT-B ImageNet-1K Top-1 from 81.1% to 81.9% and improving video classification and visual-probe question answering at additional parameter and computational cost.
Dynamic Inverse Rendering for Enhanced Material-Lighting Decomposition: DIR turns rigid motion during hand-held capture into illumination constraints, using three stages of tracking, geometry refinement, and physical inverse rendering to recover relightable assets; on synthetic diffuse objects, hand-held capture with estimated poses achieves 40.33 dB albedo PSNR versus 32.68 dB for static multiview capture with ground-truth poses.
EgoEverything: A Benchmark for Human Behavior–Inspired Long-Context Egocentric Video Understanding in AR Environment: EgoEverything samples question targets using real eye-tracking traces and combines delayed questioning, dual-agent evidence gathering, and human filtering to build an AR memory benchmark with over 100 hours of video and over 5,000 multiple-choice questions, where the best evaluated model, Gemini 1.5 Pro, achieves 63.1% accuracy, trailing human accuracy of 83.5% by 20.4 percentage points.
EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage: EgoPolice builds a benchmark of real body-worn camera footage using objective action definitions and per-second human annotation, with supervised linear probing and zero-shot five-way questions exposing fine-grained recognition and domain-transfer weaknesses; even Gemini 2.5 Pro achieves only 76.9% accuracy on one-minute questions, which does not justify autonomous deployment.
ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency: ESTANet introduces a lightweight, real-time framework for online error detection trained exclusively on error-free procedural videos, capturing both execution and procedural mistakes at 24+ FPS by exploiting prediction inconsistencies among standard and dynamic error-sensitive action detectors operating across complementary temporal context windows.
FlowPainter: Inpainting Optical Flow via Confidence-Guided Completion: FlowPainter reformulates dense optical flow generation as confidence-guided soft inpainting, where a lightweight confidence-aware network predicts reliable simple-flow priors to initialize diffusion states and inject time-decayed residual guidance, reducing denoising burden and accelerating training convergence by over \(3\times\).
Following Motion for Sequential Modeling in Video Frame Interpolation: Addressing the semantic fragmentation caused by fixed scanning orders in conventional State Space Models for video frame interpolation, MGMVFI reformulates optical flow motion trajectories into a 1D serialization mechanism for Mamba, combined with hollow-convolution contextual synthesis to restore flow failure zones, achieving state-of-the-art interpolation with linear computational complexity.
ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance: ForeSea tackles forensic search across long multi-camera surveillance videos by introducing ForeSeaQA, the first benchmark evaluating multimodal queries (image + text) with timestamped annotations, and proposing a person-centric 3-stage VideoRAG framework that substantially outperforms prior dense Video LMMs and retrieval baselines.
FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation: FreeFlow discards flow-specific inductive biases such as correlation volumes, warping, and iterative refinement in favor of a single feed-forward hierarchical transformer combining window, shifted-window, and downsampled global attention, establishing new state-of-the-art performance on Sintel, KITTI-2015, and Spring while dramatically reducing 1080p GPU memory usage.
HieDG: A Hierarchical Discrete Geometry-Guided Framework for Multi-Animal Tracking: HieDG discretizes unstable continuous geometric signals (position, scale, velocity) in multi-animal tracking into structured tokens via a two-level residual codebook, which are then fused with visual features and injected into a query-based tracker. It achieves SOTA association performance (with significant improvements in HOTA, AssA, and IDF1) on three animal tracking benchmarks (AnimalTrack, BFT, and BuckTales), and demonstrates generalizability on DanceTrack and SportsMOT.
Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding: MERIT stores ultra-long videos as short episodic records with complementary textual keys, then retrieves and filters neighboring context only after receiving a question, achieving 71.2% accuracy on EgoLifeQA—5.6 percentage points above WorldMM with the same GPT-5 solver—using a simple memory structure.
LongEgoRefer: A Benchmark for Long-Form Egocentric Video Referring Expression Comprehension: To overcome the limitation of existing benchmarks confined to short clips with dense target appearances, this paper introduces LongEgoRefer—the first video referring expression comprehension benchmark built upon 45-minute untrimmed first-person videos where targets appear in only 1.3% of frames—and demonstrates via a decoupled VLM-and-tracker baseline that temporal localization error is the primary bottleneck.
MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction: MAVFusion proposes a motion-aware sparse interaction mechanism, which decouples the cross-frame cross-modal interaction in infrared-visible video fusion into two paths based on optical flow-guided motion region detection: "weak interaction for static backgrounds + sparse global attention for dynamic regions." While achieving SOTA fusion quality across three benchmarks, it reduces computational complexity to approximately 5% of existing video fusion methods.
Modality-Aware Out-of-Distribution Detection for Multi-Modal Action Recognition: Addressing the limitation that existing multimodal action recognition OOD detection methods only apply regularization during training while discarding cross-modal relations at inference via off-the-shelf unimodal detectors, this paper introduces the first post-hoc modality-aware OOD detector. By capturing linear mapping residuals between unimodal and multimodal predictions, modeling low-variance linear invariants in joint feature space, and fusing them into normalized virtual logits, it establishes a new state of the art on the Multi-OOD benchmark.
O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning: Addressing the failure of holistic vision-language models to capture subtle physical transformations and process violations in industrial video anomaly detection, O-VAD proposes a training-free agentic framework that tracks per-object spatiotemporal state evolution and conducts a six-step cognitive reasoning process, achieving state-of-the-art anomaly localization and root-cause analysis without domain-specific fine-tuning.
On the real-world generalisability of Optical Flow models: This paper introduces an 8,204-sample real-world optical flow evaluation benchmark featuring FlowFactor, an isolated four-factor challenge dataset, demonstrating that accuracy gains on modern synthetic benchmarks have decoupled from real-world performance while exposing an inherent tension between large-displacement tracking and lighting robustness.
Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning: Addressing the severe inference latency and cross-event interference in standard token-by-token dense video captioning, PadCaptioner restructures the causal dependency graph via latent global planning and achieves lossless parallel autoregressive decoding with a 3.8× actual wall-clock speedup using event-factorized attention.
Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach: To tackle substantial joint visibility dropout caused by restricted camera views in mobile robotics, wearable platforms, and crowded environments, this paper introduces PartialVisGraph—the first hypergraph framework dedicated to skeleton action recognition under constrained Field-of-View (FoV)—combining learnable virtual hyperedges, a soft incidence matrix, and visibility prior gating to achieve up to 68.8% gains over strong baselines under severe view restrictions.
QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understanding: To resolve biased relevance estimation, limited visual diversity, and temporal collapse in long and streaming video understanding, QSVideo decouples query intent into Object/Action/Location semantic importance via QSRanker and leverages QSRetrieval with L2 visual distance and adaptive temporal windowing to achieve state-of-the-art video QA under strict frame budgets.
ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding: Addressing the causality violations in offline video anomaly understanding and the transient feature dilution and prohibitive compute costs in streaming models, ReactVAU proposes a Slow-Fast decoupled framework that employs Spatial Grid Folding for continuous fast filtering and protects critical cues with Anomaly-Aware Persistent Memory, reactively awakening a dormant heavyweight MLLM solely for suspicious events.
RT-RMOT: A Dataset and Framework for RGB-Thermal Referring Multi-Object Tracking: To overcome the failure of conventional RGB-based referring multi-object tracking under low-visibility conditions like nighttime and smoke, this paper proposes the new RGB-Thermal RMOT (RT-RMOT) task, establishes the benchmark dataset RefRT, and designs RTrack, a unified MLLM framework optimized via Group Sequence Policy Optimization with Clipped Advantage Scaling.
SAFER-Activities: A Dataset for Smart Assessment of Fall Events and Routine Activities: SAFER-Activities establishes an untrimmed multi-camera benchmark spanning over 66 hours, 85,310 frame-level action instances, and 5,406 falls with a specialized wheelchair subset, revealing remarkable skeleton-model robustness against severe domain shifts alongside the catastrophic degradation of frozen pretrained RGB backbones.
ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding: ScanFocus proposes a coarse-to-fine spatio-temporal video grounding framework that decouples the task into a global spatio-temporal scan and a local boundary focus, combining lightweight deformable semantic-motion fusion with a Semantic-Guided Temporal Aggregator (SGTA) to capture boundary cues with halved computation.
Selective Synergistic Learning for Video Object-Centric Learning: Addressing the inductive bias asymmetry where slot attention encoder maps exhibit sharp but noisy boundaries while decoder object maps yield smooth semantics with blurry contours, SSync introduces a selective mutual distillation framework with linear complexity based on spatiotemporal local consistency, combined with transitive pseudo-label merging to eliminate slot over-fragmentation.
SFDATrack: Generalized Source-Free Domain Adaptive Tracking Under Adverse Weather Conditions: SFDATrack proposes the first source-free domain adaptive framework for visual tracking under adverse weather conditions. Through a bidirectional interactive Mamba module and a hyperspherical prototype projection module within a Mean-Teacher architecture, it robustly adapts to multiple weather conditions without accessing any source domain data, achieving state-of-the-art performance on both synthetic and real-world adverse weather tracking benchmarks.
Sink-Token-Aware Pruning for Fine-Grained Video Understanding in Efficient Video LLMs: This paper identifies that high-attention but semantically void "sink tokens" under attention pruning are the critical obstacle to the collapse of fine-grained video understanding. The authors propose SToP, which defines a cross-frame sink score to quantify the sink tendency of tokens. This score is integrated into spatial pruning (STSP, to lower the priority of retaining sink tokens) and temporal pruning (STTP, to increase the pruning probability of sink tokens). At an extreme retention rate of 10%, SToP significantly shrinks the performance loss on hallucination and compositional reasoning tasks (e.g., reducing the loss of FastVid from 15.69% to 6.32%, and VisionZip from 16.79% to 6.87%).
SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos: Addressing the joint timestamp localization and execution quality grading (good vs. tip) in untrimmed ego-exo videos, SkillSpotter integrates bidirectional cross-view attention, gated 3D body pose fusion, and adaptive temporal suppression, boosting class-specific mAP from 12.40 to 21.82 and achieving 94% of the empirical human inter-annotator agreement ceiling in balanced accuracy.
STEP: Score-Based Temporal Energy for Human Pose Video Anomaly Detection: Addressing the structural collapse caused by noise destroying kinematic constraints when expanding temporal windows in skeleton-based video anomaly detection, STEP projects human pose sequences into a compact, whitened principal component space for multi-scale denoising score matching with sequence-level confidence weighting, achieving 90.1% AUROC on UBnormal (+12.2% gain) at 1000+ FPS throughput.
SVI360: Spherical Video Interpolation: To tackle severe polar distortions and correspondence failures under large displacements in equirectangular projection, SVI360 introduces a dual-branch coarse-to-fine framework that leverages rotated orthogonal views to iteratively refine optical flow and intermediate features, achieving state-of-the-art omnidirectional video frame interpolation.
Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioning: To tackle modality detachment and temporal incoherence in audiovisual video captioning, this paper introduces TCA-Captioner, an omnimodal captioner trained via an iterative Observer-Checker-Corrector (OCC) data refinement pipeline on high-density human interaction videos, and proposes TCA-Bench with a Decoupled Evaluation Protocol to systematically isolate cross-modal binding and temporal causal reasoning.
Towards Long-Form Spatio-Temporal Video Grounding: This paper formalizes and explores the Long-Form Spatio-Temporal Video Grounding (LF-STVG) task for the first time. It proposes ART-STVG, a memory-augmented autoregressive Transformer architecture that processes videos frame-by-frame as streaming inputs. By employing spatial/temporal memory banks with selective retrieval strategies and a cascaded spatio-temporal decoding design, ART-STVG significantly outperforms existing methods on 1 to 5-minute long video grounding tasks without relying on additional long-form video training data.
Towards Video Anomaly Detection from Event Streams: A Baseline and Benchmark Datasets: Addressing benchmark scarcity and sparse asynchronous modeling hurdles in event-based video anomaly detection, this paper introduces large-scale synchronized event-stream benchmarks along with EWAD, an event-centric baseline framework integrating density-aware dynamic sampling, density-modulated attention, and RGB-to-event knowledge distillation.
ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits: To overcome the severe temporal blindness in video-language foundation models caused by permutation-invariant pooling and static shortcuts, this paper introduces Cross-Modal Temporal Edits (XTE) to synthesize strictly controlled cross-modal hard negatives, paired with ViTAL-X—a lightweight spatiotemporal adapter with dual-modality LoRA that outperforms 7B models using only 0.4B parameters and 1M clips.
What You Ask is What You Ground: Bridging Question Intent to Temporal Evidence for Grounded VideoQA: To tackle question-invariant grounding collapse where models predict nearly identical temporal intervals across distinct questions, GroundFormer introduces learnable communication tokens to inject question intent into video features prior to localization, combined with factorized MIL cross-attention and hierarchical contrastive learning under candidate-level supervision alone.
Why Can't I Open My Drawer? Mitigating Object-Driven Shortcuts in Zero-Shot Compositional Action Recognition: Addressing the object-driven shortcut where models guess verbs based on static object classes due to asymmetric learning difficulty and sparse supervision in zero-shot compositional action recognition, this paper proposes RCORE—combining Co-occurrence Prior Regularization (CPR) to synthesize unseen compositions and penalize frequent hard negatives with Temporal Order Regularization for Composition (TORC) to enforce temporal sensitivity—achieving substantial gains on unseen compositions.
WiFlow: Estimating Optical Flow using WiFi Channel State Information: WiFlow introduces the first framework to estimate dense 2D optical flow fields solely from commodity WiFi Channel State Information (CSI), achieving robust cross-subject motion reconstruction in privacy-sensitive and unlit indoor environments through antenna quotient normalization and a decoupled flow-mask architecture.
A Dual-Transformer Architecture with Cross-Attention for Multi-Camera View Recommendation: The paper separates the encoding of previously shown footage from the evaluation of current camera candidates, letting candidates query temporal memory through cross-attention; with frozen SwinV2-Tiny features and focal loss, it reaches 69.65% [email protected] on TVMCE, a thresholded recommendation metric rather than forced six-way selection accuracy.
Aligning Anything: Hierarchical Motion Estimation for Video Frame Interpolation: The paper injects SAM object regions into contextual feature extraction, flow distillation, and feature supervision, retaining pixel-level motion flexibility while improving object-level coherence; under matched retraining settings, AMT-G improves from 36.42 to 36.62 dB on Vimeo90K.
Amplify, Aggregate, and Adjust: VideoMAE-based Holistic-Subtle Aggregation for Micro-Action Recognition: A3-MAE extends an RGB-only VideoMAE by locating and amplifying subtle-motion regions, exchanging holistic and local information bidirectionally, and calibrating distillation supervision with instance-wise confidence, improving MA-52 Action Top-1 from 66.47% to 67.92% and iMiGUE from 66.44% to 67.08%.
Bayesian Uncertainty Attribution-Guided Fine-Tuning for Open-Set Action Recognition: The paper turns ensemble uncertainty from a test-time rejection score into a representation-refinement signal, using three-stage fine-tuning and two-phase distillation to reach 88.50 AUROC with a single TPN student on UCF101/HMDB51, compared with 88.62 for the refined ensemble.
ChronusOmni: Improving Time Awareness of Omni-Modal Large Language Models: ChronusOmni interleaves absolute-time text, video frames, and corresponding audio for Ola-7B, then learns six temporal tasks through audiovisual dense-captioning SFT and task-reward GRPO, reaching 79.85 audio-to-time [email protected] on ChronusAV and 34.5 zero-shot temporal-grounding mIoU on LongVALE.
CLUE-VAD: Structured Semantic Clues for Understanding Explainable Events in Video Anomaly Detection: CLUE-VAD organizes video evidence into Action, Environment, and Object captions, performs weakly supervised detection through category-aware fusion, and generates explanations using segment attention and a clue-conditioned language model, achieving 89.47% AUC / 88.24% AP on UCF-Crime / XD-Violence without establishing superiority over every multimodal detector.
CTEPM: Continuous-Time Event Process Memory for Long-Video Language Models: CTEPM represents long videos as continuous-time event processes with semantic marks and conditional intensities, computes temporal evidence before generating an answer, and improves LongVideoBench accuracy from 46.0% to 49.2% under the same 64-frame budget.
DART: Deformable Adaptive Reasoning with Temporal Queries for Online Skeleton-Based Action Recognition: DART uses a Deformable Multi-scale Temporal Network to locate informative historical skeleton features and learnable queries to distill action evolution cues from compact memory, achieving per-frame recognition without explicit future forecasting and reaching 73.99% AUC on NTU60 X-Sub, 3.04 percentage points above InfoGCN++.
DE2TR: Dual Evidence Detection Transformer for Video Temporal Grounding: DE2TR assigns semantic relevance and temporal endpoints to different query subspaces, combining dual-prior guidance and boundary augmentation to improve average mAP on the QVHighlights test split from TD-DETR's 46.69% to 49.87%, a gain of 3.18 percentage points.
DisentangledTMR: Privacy-Preserving Skeleton Motion Retargeting via Factorized Transformers: DisentangledTMR separately encodes a source person's action and a reference person's skeletal identity, then recombines them through a factorized Transformer; at its recommended partial-retargeting setting on NTU60, pre-trained re-identification falls from 75.4% to 18.1%, with 75.8% pre-trained action recognition and 87.1% action recognition after retraining.
EatVid-Bench: A Multimodal Fine-Grained Eating Behavior Video Dataset: EatVid-Bench turns 682 real-world eating videos into traceable hierarchical behavioral annotations and 8,184 QA pairs, shows that strong health-related descriptions can coexist with weak objective perception, and uses AGCoT-LoRA to improve Qwen2.5-VL-7B's objective weighted score from 0.1623 to 0.3214 on the 64-frame Hard subset.
Egocentric Procedure Parsing: VidParse reformulates online step recognition in egocentric video as detecting visual segments and interpreting them through a task graph without fine-tuning, achieving [email protected] scores of 80.5 on GTEA and 77.6 on EgoPER while still requiring step-annotated reference sequences to construct prototypes and the graph.
EventSTU: Event-Guided Efficient Spatio-Temporal Understanding for Video-LLMs: To tackle high computational costs and visual redundancy in long video understanding for Video-LLMs, EventSTU proposes a training-free, bio-inspired event-guided framework combining coarse-to-fine keyframe sampling with zero-cost physics-semantic token pruning, reducing FLOPs by 66.7% while consistently surpassing original baseline performance.
FeatTracker: Short- and Long-Range Temporal Feature Consistency for Robust Underwater Object Tracking: Addressing severe color distortion and low visibility in underwater visual tracking, FeatTracker shifts enhancement from pixels to the deep feature space, pairing diffusion-like refinement and dynamic memory for short-range consistency with Haar wavelet structure/detail decomposition for long-range trajectory modeling to set new SOTA performance across four UOT benchmarks.
From Noise to Events: Conditional Diffusion for Event Data Augmentation: Addressing the issues of broken temporal continuity and ambiguous labels in conventional event cropping and mixing augmentations, N2E introduces a dimension-decoupled tensor representation and a conditional DDIM diffusion framework that generates physically faithful and diverse augmented event streams by steering denoising steps, stochastic latent noise, and condition embeddings.
FusionTrack: Collaborative Multi-Object Tracking with Arbitrary Multi-UAVs: Addressing arbitrary UAV fleet dynamics, frequent target entries/exits, and severe cross-view appearance variations, FusionTrack discards the conventional decoupled track-then-associate paradigm in favor of an end-to-end bidirectional fusion framework between Tracklet Memory and Trajectory Identity Pools, paired with View-aware Hierarchical Clustering to achieve state-of-the-art collaborative tracking.
Geometry Grounding: Elevating Blind Distortion Correction with 3D Structural Priors: Addressing the 3D structural collapse caused by 2D appearance-only optimization in single-image distortion correction, G2DC leverages a pretrained 3D foundation model (VGGT) to enforce high-dimensional structural alignment and explicit physical constraints (point map collinearity, depth topology, and camera intrinsics), achieving state-of-the-art 3D-consistent rectification.
GuideMe: Benchmarking Multi-Domain Task Guidance and Intervention in Streaming Video: Constructs GuideMe, the first multi-domain streaming video benchmark for closed-loop interactive task guidance (comprising 2,458 long videos and 47,775 interaction samples), and introduces a tripartite evaluation framework combining temporal-semantic bipartite matching, intervention timing classification, and LLM-as-a-Judge to expose a critical capability asymmetry: current MLLMs deliver routine step instructions reasonably well but fail dramatically at real-time error detection and corrective guidance.
H2SVC: Head-aware Heterogeneous Streaming Video Cache for Online Video Understanding: Reveals the intrinsic temporal receptive field heterogeneity among MLLM attention heads, proposing a training-free Head-aware Heterogeneous Streaming Video Cache (H2SVC) framework that combines offline static routing with geometric trajectory curvature eviction to achieve streaming SOTA while slashing KV memory by over 50%.
MASS: Motion-Aligned Selective Scan for Refinement in Flow-Based Video Frame Interpolation: MASS redefines the feature scanning of SSM in video frame interpolation from static spatial grids to dynamic sequence modeling along optical flow trajectories. Through learnable non-linear path integration and velocity-aware adaptive scanning, it significantly improves interpolation quality under large displacement and complex occlusion scenarios while maintaining computational efficiency. On the SNU-FILM Extreme subset, it outperforms the previous state-of-the-art by 0.38 dB with 26.04 dB PSNR.
Mode-Conditioned Residual Calibration for Multi-Object Tracking: Addressing mode-dependent cost miscalibration in crowded and occluded scenes, MCRC introduces a lightweight plug-and-play module that diagnoses error regimes via cluster prototypes to apply residual cost corrections, combined with an uncertainty-gated semantic denoiser to significantly cut identity switches with negligible latency overhead.
Motion-aware Sparse Pipeline for Lightweight Object Tracking: Addressing the issues of delayed pruning caused by noisy early attention and computation waste in dense prediction heads, MaST injects a Gaussian motion prior into first-layer token scoring and deploys a score-first, regress-once sparse head, setting new SOTA for lightweight tracking at nearly double the edge inference speed.
MotionChain: Fine-Grained Video Motion Understanding via Structured Decomposition: Addressing the failure of existing Vision-Language Models (VLMs) under fixed-rate sampling to capture sub-second motion transitions and their lack of structured temporal constraints, MotionChain introduces a training-free framework that decomposes video motions into structured 4-tuples, calibrates physical transition boundaries via Farneback optical flow energy changepoints, and verifies candidate answers against segment-wise grounded visual claims.
OCTA-SOT: Online Cross-Modal Trajectory Adjustment for RGBT Anti-UAV Single Object Tracking under Spatio-Temporal Misalignment: Addressing the pervasive spatio-temporal misalignment and parallax between visible and thermal sensors in real-world drone defense, OCTA-SOT introduces a training-free, plug-and-play online cross-modal trajectory adjustment framework that dynamically estimates inter-modality mappings via Kalman filtering and rescues unconfident tracker states, improving DIMP baseline Mean AUC by 9.3% on Anti-UAV300.
ODONet: Online Dynamic Offset Network for Visual Object Tracking: Addressing the issue that existing offset-based trackers operate offline on single frames and fail to adapt to continuous target appearance deformation and motion drift, ODONet injects historical bounding box motion prompts via two-way cross-attention at the encoding stage and dynamically fuses online propagated motion offsets with current offline offsets during inference, actively steering deformable attention toward an adaptive receptive field and setting state-of-the-art results on LaSOT (75.1% AUC) and GOT-10k (81.4% AO).
PA-VAD: Pure Pseudo-Anomalous Video Anomaly Detection Based on Diffusion Models: PA-VAD proposes a framework completely free of any real anomalous videos. By leveraging CLIP-guided initial frame selection and VLM prompt refinement, it drives a video diffusion model to synthesize class-aware pseudo-anomalous segments. It designs a Domain Alignment Regularization Module (DARM) to suppress the magnitude bias of pseudo-anomalies in the feature space. PA-VAD outperforms UVAD SOTA on ShanghaiTech (98.2% AUC), UCF-Crime (82.5%), and XD-Violence (95.1%), even surpassing some WVAD methods that utilize real anomalous videos.
Path-JEPA: Path Signature Based Predictive Learning for Skeleton Action Recognition: Path-JEPA is the first framework to adopt path signatures from rough path theory as self-supervised prediction targets for skeleton action recognition, utilizing multi-scale spatial paths (joints, edges, chains) and global-local temporal views within a leak-free JEPA architecture to achieve state-of-the-art performance and intrinsic invariance to frame-rate shifts.
Progressively Spiral Mamba Fusion for Multimodal Tracking: To overcome quadratic computational complexity and the neglect of intra-modal cross-layer complementarity in multimodal tracking, PSMTrack presents a progressive SSM tracker that incorporates difference-guided bidirectional Mamba enhancement, sparse-routed cross-layer interaction, and hybrid local-global spiral Mamba fusion, achieving state-of-the-art performance across RGB-D/T/E benchmarks at 43.6 FPS.
Proposal Score Realignment Guided by Semantic Completeness for Weakly Supervised Temporal Action Localization: Addressing the severe mismatch between proposal confidence scores and true proposal quality in weakly supervised temporal action localization that degrades pseudo-label training, this paper proposes SCLR, a plug-and-play score realignment framework that constructs dual semantic centers, disentangles generic from video-specific semantics via self-supervision, and drives completeness learning to achieve a new state-of-the-art 53.2% average mAP on THUMOS14.
Remembering Across Blocks: Topology-Conditioned Block-Progressive Memory for Skeleton-Based Action Recognition: To overcome representation homogenization (over-smoothing) and the attenuation of early fine-grained spatiotemporal cues caused by repeated one-hop neighborhood aggregation in skeleton GCNs, this paper proposes a topology-conditioned block-progressive memory updated online along the block axis via self-supervised reconstruction surprise, achieving state-of-the-art results across major benchmarks even when the memory module is completely detached at inference time (Mem-free).
Rethinking Temporal Modeling in Visual Object Tracking via Decoupled Auxiliary Supervision: Addressing the issue where temporal tokens are marginalized by dominant spatial appearance matching in visual object tracking, this paper proposes DASTrack, a framework with Decoupled Auxiliary Supervision that forces the network to localize targets solely using temporal tokens via template masking during training, eliciting genuine temporal modeling with zero inference overhead.
S3-Prune: Stability-Aware Token Budgeting for Long-Form Video-Language Models: To tackle transient observational noise and cumulative budget misallocation in long-form video pruning, S3-Prune formulates token budgeting as state estimation with Kalman filtering, sustaining full baseline accuracy even under extreme 90% token pruning.
Saber: Anchoring Semantics to Scale-Aware Kinetic Salience for Zero-Shot Skeleton Action Recognition: Saber addresses spectral smoothing in GCNs via a scale-aware bipartite encoder equipped with multi-head instance-adaptive high-pass filtering, and replaces fragile point-to-point cross-modal projections with kinetic-energy-guided variance-adaptive Gaussian distributions, establishing new state-of-the-art results across ZSL and GZSL benchmarks.
SOVTrack: Open-Vocabulary Multi-Object Tracking with Self-Supervised Pseudo Labeling and Feature Distillation: Addressing poor generalization on unseen categories and insufficient temporal modeling in Open-Vocabulary Multi-Object Tracking (OVMOT), SOVTrack introduces detection-guided bidirectional cycle consistency to extract high-quality pseudo-labels with difficulty scores, coupled with a decoupled dual-branch architecture that adversarially distills multi-cue SAM2 representations, achieving state-of-the-art tracking on TAO using only 10K raw video frames.
SPAR: A Sequential Primacy and Attribution Ranking Framework for Skill Determination: SPAR explicitly emulates the progressive temporal evolution of human evaluators' cognitive states via Stage-aware Cognitive Memory (SaCM), combined with context-sensitive Temporal Window Weighting (TWW) and a counterfactual Judgment Bias Mitigation (JBM) loss, delivering state-of-the-art pairwise skill ranking and robust absolute score prediction.
Sticking Information in Plain Sight: Encoding and Detecting Hidden Stickers in the Real World: Addressing the flaw of conventional image steganography that requires reprinting the entire image container and fails under minor physical misalignments, this work proposes an end-to-end framework jointly optimizing an encoder, localizer, and decoder to embed 100-bit messages into transparent stickers placed on physical surfaces, achieving robust detection and decoding under placement errors, perspective distortions, and multi-sticker scenarios.
SV-TAD: Native Sparse Convolutions for Efficient Temporal Action Detection: SV-TAD introduces native sparse 2D convolutions and an adapter architecture designed directly for dynamically pruned Vision Transformer token sequences, eliminating dense grid reconstruction overhead and reducing VideoMAEv2-L computation by 64% with a 2.2x inference speedup while preserving state-of-the-art detection accuracy.
Towards Effective Long Video Understanding: Dynamic MAS Construction via Meta-Agent: To eliminate the computational redundancy and structural rigidity of static multi-agent architectures in long video understanding, DyMAC introduces a Vision-Language Meta-Agent that generates instance-specific multi-agent systems via executable code compiled into directed computational graphs, optimized through SFT and Reinforcement Learning with Verifiable Rewards (RLVR) for superior accuracy and efficiency.
TR-MoE: Temporal Reliability-Aware Mixture-of-Experts for Robust Tracking: Addressing the complementary nature of SAM2's fine-grained segmentation and discriminative trackers' global semantic robustness under occlusion, TR-MoE reformulates multi-tracker fusion as a temporal sequence modeling problem, employing a Mamba-based trajectory refiner and a KAN-enhanced disentangled head to achieve state-of-the-art results on LaSOT and GOT-10k.
UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation: Addressing the lack of motion priors and susceptibility to motion blur in single-frame driving extrapolation, UniDynamics introduces a diffusion-based framework that leverages a single event-RGB pair to simultaneously generate future RGB video, 3D metric depth, and optical flow fields via event latent enhancement and a decoupled perceptual dynamics space.
Where and What: Long-Term Object Tracking in Egocentric Videos: Whareformer presents the first learning-based framework for the Out of Sight, Not Out of Mind (OSNOM) task in egocentric videos, projecting relative 2D appearance and 3D spatial distances into learned likelihood embeddings and employing an explicit New Track token within a Transformer encoder to achieve 72.2% mPCL and 84.0% IDF1 on EPIC-KITCHENS-100.