Skip to content

๐Ÿง‘ Human Understanding

๐Ÿง  NeurIPS2026 ยท 31 paper notes

๐Ÿ“Œ Same area in other venues: ๐ŸŽž๏ธ ECCV2026 (171) ยท ๐Ÿ“ท CVPR2026 (218) ยท ๐Ÿ”ฌ ICLR2026 (55) ยท ๐Ÿงช ICML2026 (7) ยท ๐Ÿค– AAAI2026 (24) ยท ๐Ÿง  NeurIPS2025 (28)

๐Ÿ”ฅ Top topics: Face & Gaze ร—5 ยท Diffusion Models ร—4 ยท Human Pose ร—3 ยท Speech & Audio ร—3 ยท Adversarial Robustness ร—2

AnyHand: A Large-Scale Synthetic Dataset for RGB(-D) Hand Pose Estimation

This paper introduces AnyHand, a large-scale synthetic RGB-D dataset comprising 2.5M single-hand and 4.1M hand-object interaction images equipped with diffusion pose priors, physics-simulated grasps, full 3D forearm context, and aligned depth maps, which consistently improves state-of-the-art RGB models (HaMeR and WiLoR) on both in-domain and zero-shot cross-domain benchmarks without architectural modifications and sets a new RGB-D SOTA via a lightweight bidirectional depth-fusion module.

AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild

This paper proposes AnyMo, a geometry-aware and setup-agnostic framework for human motion understanding that integrates dense body-surface physics simulation, masked cross-view predictive contrastive pretraining, full-body product-quantized tokenization, and multi-task contrastive instruction tuning, improving average zero-shot Accuracy, macro-F1, and R@2 across 14 unseen human activity recognition benchmarks by 11.7%, 11.6%, and 22.6% respectively, while substantially advancing cross-modal retrieval and wearable motion captioning.

Asymmetric Invertible Threat: Learning Reversible Privacy Defense for Face Recognition

This paper identifies the "asymmetric invertible threat" where existing input-space face privacy cloaks can be neutralized by learned inverse purification or restoration mappings, and proposes the ARFP framework integrating Key-Conditioned Manifold Binding (KMB), Adversarial Restoration-Aware Training (ARAT), and Authorized Reversible Restoration (ARR), achieving a 95.91% mean Protection Success Rate (PSR) across 8 black-box FR models, 91.2% PSR under the strong paired BINN restoration attack, high-fidelity authorized recovery (31.2 dB PSNR, 0.94 SSIM), and nonce-based tamper indication.

AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking

AVI-HT addresses severe finger occlusion during egocentric hand-object interactions by coupling a two-level hierarchical cross-sensor attention mechanism with kinematic prior masks to fuse 12-channel on-glove 6-DoF IMU measurements with egocentric imagery, decreasing mean keypoint position error by 16.1% on DexGloveHOI.

BiMoGen: Bidirectional Motion-Text Generation via Unified Masked Discrete Diffusion

BiMoGen combines a unified motion-text vocabulary with a bidirectional masked diffusion model for motion generation and captioning, using correspondence learning before conditional generation and generation-aware self-correction to achieve T2M R@1 of 0.555 and M2T CIDEr of 60.2 on HumanML3D, without outperforming existing unified models on every metric.

Breaking the Group Size Barrier: Parameter-Efficient Group Dance Generation with Chain-of-Dancers

ChainDance reformulates music-driven group dance generation as a sequential conditional Chain-of-Dancers, building on a frozen single-dancer diffusion backbone with lightweight role/motion encoders and training-free test-time noise optimization to eliminate group-size rigidity and identity swapping while cutting training time and parameters by 3โ€“6ร—.

Complexity-guided Regularization for Generalizable Human Gaussian Splatting

To tackle blurry rendering artifacts in high-frequency intricate regions for generalizable human Gaussian splatting, this paper establishes a multi-scale complexity prior from local geometric and textural Shannon entropies of a clothed human mesh, regularizes Gaussian density via pairwise monotonic ranking, and samples complexity-guided local patches for supervision, achieving state-of-the-art novel view synthesis at 34 FPS.

EgoForce: Robust Online Egocentric Motion Reconstruction via Diffusion Forcing

EgoForce addresses the challenge of online full-body motion recovery from sparse, noisy egocentric head and hand tracking by introducing temporally asymmetric noise scheduling inspired by Diffusion Forcing, achieving smooth, drift-free, real-time motion generation under strict causal constraints.

ELMA: Benchmarking Anaphoric Compositions for Long-Term Text-to-Motion Generation

This paper formulates the novel task of long-term text-to-motion generation under anaphoric compositions, introduces the ELMA dataset (13.2k continuous 30-second 3D motion clips with explicit cross-segment dependency annotations) alongside the target-sliced ELMABench benchmark, and demonstrates that fine-tuning existing models with historical text context injection substantially improves cross-segment state persistence and semantic alignment.

EverAnimate: Minute-Scale Human Animation via Latent Flow Restoration

EverAnimate addresses cumulative background degradation and character identity drift in long-horizon pose-guided human animation through Persistent Latent Propagation and Restorative Flow Matching with velocity rescheduling, achieving minute-scale, high-fidelity video generation via lightweight LoRA tuning.

FA-LAM: Focus-Aware Large Avatar Model for One-Shot 4D Animatable Gaussian Head

To resolve noisy cross-attention activations in occluded regions, gradient conflicts between reconstruction and animation, and quadratic memory explosion in streaming inputs within feed-forward large avatar models (LAM), FA-LAM introduces bilateral-symmetry-guided semantic attention regularization, a two-stage static-then-residual disentangled training pipeline, and visibility-gated autoregressive feature fusion, achieving state-of-the-art performance across one-shot/few-shot animatable full-head avatar creation and constant-memory 4D streaming reconstruction.

FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery

FactorizedHMR first deterministically estimates the torso, shape, and camera-space trajectory, then fixes that structure while masked flow matching completes limbs, the head, and world motion; its clearest benefits concern severe occlusion and trajectory drift rather than uniformly superior conventional metrics.

GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction

GazeFlow generates whole gaze trajectories through flow matching conditioned on local video features and global task tokens, reaching F1 0.491 and average angular error 9.01 on EGTEA Gaze+ while improving displacement statistics, but its default model uses future frames and is not directly an online eye tracker.

GRASP: Learning to Ground Social Reasoning in Multi-Person Non-Verbal Interactions

The paper introduces GRASP, the first large-scale grounded social reasoning benchmark that bridges high-level social QA with fine-grained gaze and gesture events (290K QA pairs across 46K videos), and proposes the Social Grounding Reward (SGR) to discipline MLLM reasoning traces toward verified interaction participants under GRPO, outperforming proprietary front-runners while eliminating ungrounded deliberation hallucinations.

GRASP: Learning to Ground Social Reasoning in Multi-Person Non-Verbal Interaction

This paper introduces GRASP, a large-scale multimodal social reasoning benchmark (290K QA pairs across 749 hours of multi-person videos) that systematically ties high-level social QA to fine-grained gaze and deictic gestures, alongside Social Grounding Reward (SGR)โ€”a participant-level RL signal that discourages spurious reasoning shortcuts and substantially boosts evidence-grounded social reasoning.

HOPE: Hand-Object Pressure Estimation from Monocular Videos

HOPE formulates hand-object pressure estimation as predicting temporally evolving per-vertex normal pressure and contact directly on the MANO hand mesh from monocular RGB video, leveraging a unified multi-source supervision pipeline and a contact-gated vertex video transformer to achieve tactile-free physical interaction perception.

HumanScore: Benchmarking Human Motions in Generated Videos

This paper presents HumanScore, the first systematic benchmark for evaluating the biomechanical plausibility of human motions in AI-generated videos; built upon a three-tier anatomical, kinematic, and kinetic hierarchy with six interpretable metrics across 102 standardized motion prompts, it evaluates 13 state-of-the-art video generators and exposes a clear gap between pixel-level visual realism and physical/physiological fidelity.

Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment

This paper introduces a unified sign language understanding model that integrates a lightweight decoupled pose-lip visual backbone, a Sliding Perceiver mapping network aggregating local lexical visual context, and a multi-task training curriculum with soft-decoding alignment, achieving state-of-the-art performance on both sign language translation (SLT) and sign-subtitle alignment (SSA) across BOBSL and How2Sign.

OpenVTON-Bench: A Large-scale High-Resolution Benchmark for Controllable Virtual Try-on

Addressing studio-centric bias, sub-megapixel limitations, and semantic blindness in existing virtual try-on benchmarks, this paper introduces OpenVTON-Bench comprising ~100K 1.5K-resolution image pairs and establishes a hybrid evaluation protocol integrating five-dimensional VLM scoring with SAM3-guided multi-scale representation metrics.

Past as State, Present as Attention: Persistent-State Blockwise Flow Matching for Long-Horizon Co-Speech Motion Generation

Addressing the trade-off between style drift and ballooning computational costs in long-horizon co-speech motion generation, PASPA compresses past history into an inter-block persistent state while modeling current local dynamics with intra-block bidirectional self-attention, achieving state-of-the-art long-sequence consistency and \(\mathcal{O}(1)\) step cache complexity.

PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognition

Instead of adding an RGB action-recognition branch, PoseBridge preserves visual semantics inside pose estimation before they are compressed into joint coordinates, then transfers them through skeleton-conditioned bridging and semantic prototype adaptation, improving over the strongest compared baseline by 13.3โ€“17.4 percentage points across eight Kinetics-200/400 splits.

Re:Cognize: Open-Set Comic Character Re-Identification

Re:Cognize separates identity emergence from identity maintenance through four gallery protocols over the same reading-order stream, finds that growth under predicted labels harms recognition with random seeds, and uses Re:Cast's character averages, page evidence, and cross-page binding to recover part of the benefit of correctly labelled updates under explicit conditions.

ReSCUE: Re-translation with Sentence Commitment for Unsegmented Long-Form Simultaneous Sign Language Translation

ReSCUE is the first unified framework for simultaneous sign language translation (SLT) on unsegmented long-form video streams, combining inference-aware training for partial inputs and non-signing pauses, stabilized re-translation with biased search and suffix masking to suppress visual flicker, and a punctuation-driven sentence commitment mechanism with token arrival time tracking for online segmentation, approaching oracle offline quality at 0.7โ€“1.6s latency across sentence-level and newly curated long-form benchmarks (How2Sign-Long and CSL-Story).

Seeing Speech: Learning Visible Articulatory Dynamics for Speech-Driven 3D Facial Animation

Seeing Speech learns facial motion along spreading, opening, and protrusion coordinate directions, retrieves local motion features through a speechโ€“articulatory memory, and composes them into 3D mesh animation through topology-aware residuals, reducing facial and lip reconstruction errors on VOCASET and TFHP without recovering the physiological mechanisms of actual speech organs.

Semantic Motion Anchors: Bridging Motion and Meaning in Co-Speech Gestures

This paper proposes transforming 3D co-speech gestures into natural-language "Semantic Motion Anchors" that capture both physical form and communicative intent to serve as modality-matched auxiliary contrastive supervision during training, improving text-to-gesture R@1 by 8.2% over a direct contrastive baseline on BEAT2 and achieving a 72.2% vs. 27.8% user preference win rate over RAG-Gesture in downstream retrieval-augmented gesture synthesis.

SimpliHuMoN: Simplifying Human Motion Prediction

This paper proposes SimpliHuMoN, a minimalist self-attention Transformer architecture that concatenates historical motion context with learnable future queries for bidirectional self-attention interaction and regresses multiple future motion hypotheses in a single forward pass, surpassing specialized state-of-the-art methods across 3D pose prediction, trajectory forecasting, and combined prediction without task-specific architectural modifications while achieving up to \(1.8\times\) faster inference throughput.

ST-DiffEye: Diffusion-based Continuous Gaze Generation via Joint Scanpath-Trajectory Modeling

This paper proposes ST-DiffEye, a joint scanpath-trajectory diffusion framework that aligns discrete scanpaths with continuous gaze trajectories along a shared time axis via a linearly interpolated fixation index channelโ€”achieving multi-scale gaze modeling with zero architectural overheadโ€”while introducing a Continuous Ranked Probability Score (CRPS) evaluation protocol that jointly measures accuracy and diversity to achieve state-of-the-art performance across visual search and free-viewing benchmarks.

StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video

Addressing catastrophic drift from out-of-view hands and severe interaction occlusions in egocentric video, StableHand introduces a quality-aware flow-matching framework that decomposes observation confidence into a four-channel signal (wrist and fingers per hand), seamlessly unifying reliable observation anchoring and prior-driven motion inpainting via adaptive noise scheduling, velocity zeroing, AdaLN modulation, and matched ODE initialization.

STRIDE: Automated Evaluation of Text-to-Trajectory Alignment across Diverse Contexts

STRIDE precompiles textual crowd contexts into behavioral questions, deterministic measurement functions, and expected ranges, then checks generated trajectories against them; its benchmark contains 936 scenarios, Text-Crowd achieves an overall score of 0.645, and humans agree with benchmark answers on 80% of pairs, although the latter result comes from a limited annotation subset.

Towards Unified Dynamic Face Landmark Detection

The paper describes landmarks using a face part and a normalized sequence position, then uses image-conditioned queries and iterative decoding to train one model across annotation formats and predict landmarks on demand; the unified ViT-B achieves full-set NME of 4.05, 2.80, and 1.02 on WFLW, 300W, and AFLW-19, respectively, offering unified training and interfaces rather than superiority on every metric.

TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation

TrioPose establishes a native triple-stream diffusion transformer framework (TSPA-DiT) built upon SD3.5M that treats human pose as an independent modality with zero-initialized dual-residual injection, learnable 5-state relational bias attention masks, and pose-guided spatial loss weighting, achieving a 30% AP leap in complex multi-person generation.