Decoding Children’s Gait Behavior¶

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/pediamedai/ChildrenGait
Area: Medical Imaging
Keywords: Pediatric Gait Analysis, Edinburgh Visual Gait Score, Video Foundation Models, Kinematic Prompting, Mask-Guided Pruning

TL;DR¶

To overcome the subjectivity of pediatric neuromuscular gait screening and the inability of existing adult-centric foundation models to resolve subtle motor anomalies, this paper introduces the first open-source pediatric gait video dataset CGV (110 subjects, 1,185 multi-view videos with bilateral EVGS items) and develops ChildGait-Video, an end-to-end video architecture incorporating token-level kinematic prompts and mask-guided patch pruning that reaches an average accuracy of 84% across all 34 clinical scoring items.

Background & Motivation¶

Quantitative gait analysis is a critical diagnostic pillar and rehabilitation metric for developmental and neuromuscular disorders in pediatric populations, such as cerebral palsy (CP) and hemiplegia. With the continuous progression of Brain-Computer Interfaces (BCI) and neuroprosthetics, objective gait quantification has increasingly served as a key benchmark for evaluating functional motor recovery in young patients. Currently, the clinical identification of atypical gait patterns still depends primarily on subjective visual inspection by pediatricians during routine office visits. While marker-based 3D motion capture systems and wearable Inertial Measurement Units (IMUs) can provide reliable biomechanical metrics, their deployment is strictly limited by significant equipment and space costs, which inherently biases advanced care toward well-resourced medical centers. Furthermore, attaching physical sensors directly to a child's body frequently causes sensory discomfort, poor compliance, and measurement reactivity that alters natural ambulatory behavior. Standard multi-view RGB video analysis thus represents an appealing, non-intrusive, and accessible alternative for scalable screening.

However, existing computer vision gait models have been almost exclusively constructed around adult subjects under the core assumption that gait is a mature, periodic, and dynamically stable movement. In reality, pediatric motor development is characterized by high entropy, significant intra-class variation, and non-linear anthropometric scaling in limb proportions and center-of-mass positions. Directly deploying adult-pre-trained gait recognition models or multimodal large language models (MLLMs) to pediatric populations triggers severe anatomical domain shifts and temporal blind spots. Moreover, clinical video recordings of young children are subject to heavy background clutter and frequent inter-person occlusions from parents or therapists providing physical support and encouragement. Crucially, clinical gait evaluation is an inherently phase-dependent, fine-grained task requiring precise joint angle estimation at exact moments within the gait cycle (e.g., maximum knee flexion during swing). Existing general vision-language models only perform coarse semantic categorization and lack sensitivity to subtle sub-second kinematic deviations.

To tackle the concurrent scarcity of standardized pediatric gait data and dedicated vision architectures, this work establishes a rigorous clinical video benchmark and explores an end-to-end mapping pipeline from raw RGB video to standardized clinical gait scores. Core idea: build the Children Gait Video (CGV) dataset comprising synchronized multi-view 60 FPS videos with frame-level 2D poses, segmentation masks, and expert Edinburgh Visual Gait Score (EVGS) annotations, and introduce ChildGait-Video, an end-to-end adaptation framework that renders 2D skeletal keypoints directly onto RGB frames as token-level kinematic prompts and applies mask-guided patch pruning to force spatial self-attention onto foreground patient kinematics.

Method¶

Overall Architecture¶

Departing from conventional multi-stage pipelines that perform keypoint tracking, geometric joint angle computation, and rule-based thresholding—which severely suffer from compounding error propagation and discard rich texture cues—ChildGait-Video adopts an end-to-end spatiotemporal visual mapping paradigm. The system takes multi-view RGB video streams (coronal or sagittal views) downsampled to a constant 30 FPS. During training, \(M=4\) temporal windows are randomly cropped and uniformly sampled at \(T=16\) frames per window; during deterministic evaluation, a single central window of \(T=16\) frames is extracted. In preprocessing, 2D anatomical keypoints estimated by Sapiens-2B are overlaid via Token-Level Kinematic Prompting (TKP), and instance masks generated by SAM 3 guide deterministic foreground Mask-Guided Patch Pruning (MPP). The retained foreground patch tokens are fed into a spatiotemporal ViT backbone (VideoMAE v2 architecture), followed by spatiotemporal global average pooling and a linear classification head predicting the bilateral EVGS items.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-view Pediatric Gait Video<br/>Coronal/sagittal views, downsampled to 30 FPS"] --> B["Token-Level Kinematic Prompting (TKP)<br/>Overlay 2D skeletal topology onto RGB frames"]
    B --> C["Mask-Guided Patch Pruning (MPP)<br/>Filter out background tokens via SAM 3 masks"]
    C --> D["Spatiotemporal ViT Video Encoder<br/>Spatio-temporal self-attention pre-trained on Kinetics-710"]
    D --> E["Spatiotemporal Global Average Pooling & MLP Head<br/>Aggregate foreground tokens into classification logits"]
    E --> F["EVGS Clinical Score Predictions<br/>Bilateral 34-item typical vs. atypical classification"]

Key Designs¶

1. Token-Level Kinematic Prompting: embedding anatomical priors into visual tokens
Pre-trained video foundation models possess broad motion perception capabilities but lack explicit anatomical inductive biases tailored to pediatric skeletal proportions, making them insensitive to subtle distal deviations. Conversely, purely coordinate-based representations discard muscle contours and foot-floor interaction textures. To resolve this dilemma, the framework extracts 2D keypoints \(K_t\) and their kinematic connectivity graph using Sapiens-2B, verifies them manually, and renders them directly onto the raw RGB frame \(I_t\) via a rendering operation \(R(\cdot)\), formulating prompted frames \(\hat{I}_t = R(I_t, K_t)\). When patched and projected into the Vision Transformer embedding space, tokens covering the rendered joints and limbs encapsulate both explicit anatomical topology and underlying appearance features, acting as token-level kinematic prompts that guide self-attention layers toward clinically critical kinematic axes.

2. Mask-Guided Patch Pruning: background clutter suppression and efficient fine-tuning
Pediatric clinical examinations inevitably involve complex visual distractors, such as walking aids, examination beds, and accompanying adults walking alongside the child. Standard full-frame fine-tuning easily overfits to these dynamic non-target visual cues. Capitalizing on the intrinsic sparsity of Vision Transformers, the authors utilize SAM 3 to obtain binary subject masks \(M \in \{0, 1\}^{H \times W}\) and implement deterministic spatial patch pruning. After partitioning the prompted frame \(\hat{I}_t\) into a non-overlapping patch grid, only patches whose corresponding regions in \(M\) contain sufficient foreground pixel mass are retained. This drops non-informative tokens such that \(N_{\text{foreground}} \ll N_{\text{total}}\), compelling the self-attention mechanism to focus exclusively on the child's body while significantly reducing memory and computational footprint during fine-tuning.

3. Clinically Grounded End-to-End Scoring Objective
To account for full-body kinematic synergy rather than assessing isolated joint angles, the final Transformer output tokens \(X \in \mathbb{R}^{B \times N_{\text{foreground}} \times D}\) are aggregated via spatiotemporal global average pooling into clip-level representations and projected into 2D logits via a linear head:

\[ \mathbf{L} = \mathbf{W}_{\text{head}} \left( \frac{1}{N} \sum_{i=1}^{N} \mathbf{x}_i \right) + \mathbf{b}_{\text{head}} \]

where \(\mathbf{W}_{\text{head}} \in \mathbb{R}^{2 \times D}\). To alleviate the severe class imbalance caused by the natural clinical scarcity of marked deformities (score 2 in the 3-point EVGS scale), the task is framed as binary classification (merging moderate deviation 1 and marked deviation 2 into atypical vs. 0 for typical). The model is optimized using Soft Target Cross-Entropy Loss with prior class balancing weights \(q = (\lambda, 1 - \lambda)\):

\[ \mathcal{L}_{\text{video}}(\boldsymbol{\theta}) = - \sum_{k \in \{0, 1\}} q_k \log \left( \text{Softmax}(\mathbf{L}(\boldsymbol{\theta}))_k \right) \]

Loss & Training¶

The framework adopts a two-stage transfer learning paradigm: the encoder is first pre-trained on Kinetics-710 to establish robust human action dynamics, followed by end-to-end supervised fine-tuning on the CGV dataset. Optimization is conducted using AdamW with a base learning rate of \(\eta = 10^{-4}\), weight decay of 0.05, and a linear warm-up over the first 5% of training epochs across 10 NVIDIA L40S GPUs (batch size 8 per GPU).

Key Experimental Results¶

Main Results¶

All models are evaluated on the CGV benchmark using a strict subject-level 6:1 train/test split based on unique Patient IDs, guaranteeing that all gait cycles of any given subject reside exclusively in one partition. The benchmark compares zero-shot multimodal foundation models, fine-tuned video VLMs, dedicated adult gait recognition networks, skeleton-based models, and video foundation architectures across all 34 bilateral EVGS items.

Method	Pre-training Dataset	Left Limb Accuracy L-AVG (%)	Right Limb Accuracy R-AVG (%)	Left Limb F1 (L-F1)	Right Limb F1 (R-F1)
Gemini 3 Pro (Zero-shot)	-	53	55	-	-
GPT-5.2 (Zero-shot)	-	53	56	-	-
Qwen3-VL-235B (Zero-shot)	-	54	59	-	-
Qwen3-VL-4B (Fine-tuned)	CGV	53	53	-	-
GaitBase	GREW	53	56	-	-
BiggerGait (SOTA Gait Recognition)	CCPG	54	53	0.53	0.51
ScoNet (Skeleton-based Baseline)	Scoliosis1K	68	70	0.67	0.69
VideoMAE v2 (Direct Fine-tuning)	Kinetics-710	69	72	0.70	0.71
ChildGait-Video (Ours)	Kinetics-710	84	84	0.83	0.83

Note: In McNemar's statistical significance test against VideoMAE v2, ChildGait-Video yields \((b, c) = (28, 6), p = 2.3 \times 10^{-4} < 0.001\), proving substantial statistical significance. Expert pediatrician human baseline reaches 93.8% accuracy (inter-rater ICC = 0.93).

Ablation Study¶

The ablation experiments isolate the effects of the input temporal resolution, the Token-Level Kinematic Prompting (TKP), and the Mask-Guided Patch Pruning (MPP):

Config / Variant	Frames (T)	Left Accuracy L-AVG (%)	Right Accuracy R-AVG (%)	L-F1 / R-F1	Note
ChildGait-Video (Full Model)	16	84	84	0.83 / 0.83	Full TKP + MPP setup
VideoMAE v2 Base	16	69	72	0.70 / 0.71	Baseline without TKP and MPP
VideoMAE v2 + TKP	16	72	74	0.72 / 0.73	Kinematic skeletal prompts only
VideoMAE v2 + MPP	16	78	79	0.77 / 0.78	Mask-guided spatial patch pruning only
VideoMAE v2 + Random Mask	16	68	70	0.69 / 0.69	Random masking drops critical joints (-1~2%)
VideoMAE v2 + Bounding Box Mask	16	75	77	0.74 / 0.76	Bounding box coarse background pruning
Frame ablation \(T=8\) (L/R-IC item)	8	74.1	74.1	0.73	Span 0.26s, misses kinematic sub-phases
Frame ablation \(T=16\) (L/R-IC item)	16	87.0	87.0	0.89	Span 0.53s, captures phase transitions
Frame ablation \(T=32\) (L/R-IC item)	32	90.7	90.7	0.92	Span 1.06s, modest gain with quadratic compute

Key Findings¶

Multimodal LLMs suffer from temporal motion blindness: Advanced MLLMs with hundreds of billions of parameters (e.g., GPT-5.2, Qwen3-VL-235B) achieve zero-shot accuracy hovering between 50% and 59%, barely above random chance for binary scoring. Because they are trained on causal language modeling over discrete tokens, they capture high-level semantic abstractions rather than continuous physical biomechanics; supervised fine-tuning of Qwen3-VL-4B yields marginal changes (+1% to +2%), proving that current MLLM representations do not naturally transfer to clinical gait kinematics.
Biometric gait identification differs fundamentally from clinical scoring: SoTA adult gait recognition models like BiggerGait and GaitBase collapse to 53%~56% accuracy on CGV. Their architectures prioritize identity-preserving spatiotemporal representations that discard localized, transient kinematic deviations crucial for pathology assessment.
Synergistic effect of background pruning and kinematic prompts: Introducing MPP alone delivers a significant +9% gain by removing hospital background noise and adult accompaniment; compounding MPP with TKP further elevates accuracy to 84%, confirming that explicit skeletal topologies effectively compensate for missing pediatric anatomical priors.
Pediatric age stratification: Evaluating the model on younger children (\(\le 8\) years) versus older children (\(\ge 8\) years) yields average accuracies of 82.4% and 85.2%, highlighting that immature motor patterns in younger children exhibit greater irregularity and present a higher decoding challenge.

Highlights & Insights¶

Fusing anatomical topology directly into visual token embeddings: Rendering 2D skeletal graphs directly onto RGB frames as Token-Level Kinematic Prompts (TKP) bypasses the cumulative error of geometric angle calculation while equipping the visual Transformer with explicit anatomical inductive biases at negligible computational overhead.
Deterministic spatial pruning turns denoising into an efficiency driver: Converting MAE-style patch dropping into a mask-guided deterministic filter eliminates confounding clinical background noise (e.g., medical staff, hospital clutter) while drastically slashing token volume and training compute.
Empirical demystification of MLLMs for physical-world clinical sensing: Through extensive comparative experiments, this study provides concrete empirical evidence that scaling laws in language and vision-language pre-training do not automatically confer fine-grained biomechanical understanding.

Limitations & Future Work¶

Generalization from hospital corridors to open-scene environments: Video data was collected in standardized 8-meter clinical walkways. Translating the system to real-world home environments, community health centers, and outdoor recreational spaces requires validating robustness against diverse camera perspectives, variable lighting, and everyday clothing.
Long-tail distribution of severe and rare pathological phenotypes: Although binarization mitigates extreme imbalance, rare neuromuscular disorders and atypical walking variants still suffer from limited sample representation, which could be augmented via generative physical gait simulation in future work.
Extension from 2D observational scoring to 3D joint kinetics: Currently, the system predicts discrete 2D EVGS scores. Integrating markerless 3D pose lifting to infer joint moments and ground reaction forces (GRF) represents an important frontier for comprehensive kinematic and kinetic screening.

vs. Adult Gait Recognition Models (e.g., BiggerGait, GaitBase): While adult biometric models strive for identity invariance across viewpoints and clothing, clinical scoring requires identity-agnostic sensitivity to localized, phase-specific joint anomalies. ChildGait-Video outperforms BiggerGait by approximately 30 percentage points in clinical scoring accuracy.
vs. Multimodal Large Language Models (e.g., GPT-5.2, Qwen3-VL): General MLLMs excel at high-level semantic reasoning but fail at dense sub-second temporal geometric tracking. The paper demonstrates that domain-specific spatiotemporal video adaptation is essential for quantitative clinical measurement.
vs. Pure Skeleton Action Recognition Baselines (e.g., ScoNet, SkeletonGait++): Skeleton-only models suffer from coordinate jitter and completely discard limb volume, muscle contour, and foot-ground contact information. By retaining RGB context while using poses as soft prompts, ChildGait-Video substantially exceeds skeleton baselines on distal foot contact (IC) and pelvic rotation parameters.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Establishes the first open-source pediatric gait video benchmark with expert EVGS clinical annotations and demystifies MLLM failures in fine-grained biomechanics.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluates 6 major MLLMs, 10 video and gait models, and 2 skeleton networks, accompanied by McNemar significance tests and detailed component ablations.
Writing Quality: ⭐⭐⭐⭐⭐ Rigorous narrative with crystal-clear clinical motivations, cohesive architectural design, and thorough documentation.
Value: ⭐⭐⭐⭐⭐ Provides a viable, low-cost computer vision pathway for early non-intrusive pediatric neurodevelopmental screening, holding immense clinical and societal value.