BiCE-HG: A Bi-Conditional Egocentric Hand Gesture Dataset for Intelligent Reality Systems

Conference: ECCV 2026
Paper: ECCV official page · Paper PDF
Dataset: BiCE-HG data release
Area: Human Understanding
Keywords: Egocentric Hand Gesture Recognition, Illuminance Measurement, Skeleton Sequences, Mobility Conditions, Dataset Benchmarking

Title provenance: this note retains the conference-record title. The same official PDF adds “An Indoor” to its title and lists Awfa Dakheel and Charith Abhayaratne as authors; its subject is the same indoor BiCE-HG dataset, not a separate paper.

TL;DR

BiCE-HG crosses three mobility states with two lighting configurations and provides spatial illuminance records alongside hand skeleton data; its dynamic baseline reaches 89.24% validation accuracy, but the evaluation is not participant-independent and does not establish cross-lighting generalization.

Background & Motivation

Egocentric gesture recognition observes hands through a camera that moves with its wearer, rather than a fixed camera watching an isolated hand. In AR/VR interaction, the same gesture may occur while a user sits at a desk, stands, or walks through an environment whose illumination changes with position. Skeleton-only classification does not eliminate this difficulty: dim images, motion blur, and mutual hand occlusion can corrupt the landmarks supplied by the upstream tracker. Existing resources such as EgoGesture, FPHA, and H2O cover different gesture and interaction tasks, but a scene label alone rarely specifies the measured illuminance or its spatial variation.

Labels such as “indoor” and “bright” describe an environment without making its lighting reproducible. When a model performs poorly on walking recordings, these labels cannot separate the possible effects of gesture trajectory, occlusion, blur, and illumination gradients. BiCE-HG therefore focuses on acquisition conditions and environmental measurements rather than a larger recognition architecture. Its static and dynamic skeleton baselines establish starting points, although their pooled accuracy does not replace the condition-specific analyses that the dataset could support.

The contribution is to make the capture environment inspectable, not to maximize the number of gesture classes. A head-mounted camera and a controlled indoor setting allow repeated recordings of each participant and gesture under crossed mobility and lighting conditions, accompanied by skeletons and illuminance information. Core idea: replace vague lighting descriptions with measurable experimental conditions while retaining mobility variation relevant to first-person interaction.

Method

Overall Architecture

BiCE-HG is a dataset and evaluation protocol, not a newly proposed end-to-end inference architecture. Its inputs are head-mounted GoPro gesture recordings and separate illuminance measurements of the capture space; its released outputs include static skeletons, dynamic skeleton sequences, annotations, and spatial illuminance maps. The four key designs are mobility–lighting crossed acquisition, spatial illuminance documentation, skeleton extraction and quality control, and static–dynamic evaluation. ST-GCN and its attention-enhanced variant, E-ST-GCN, serve as benchmark models rather than independent dataset contributions.

The static release provides one representative skeleton frame per recording, whereas the dynamic release preserves temporal observations. These are two representations of the same gesture task, not simply disjoint sets of static and dynamic gesture labels. The lighting records describe the dataset and support later analysis; the paper does not specify feeding lux values into the recognition baseline. Consequently, no artificial model pipeline or unverified illumination-correction module is introduced here.

Key Designs

1. Mobility–lighting crossed acquisition: cover six usage conditions for each gesture

The dataset includes 23 participants and 19 gesture classes, crossing sitting, standing, and walking with full and low lighting. Each participant performs the gestures across the six conditions, with 2,622 gesture instances repeatedly reported in the main statistics and conclusion. The vocabulary targets interface interaction, including selection, directional commands, zooming, symbolic feedback, and circular drawing, rather than long everyday activities. Crossed collection helps prevent a gesture class from appearing exclusively under one lighting configuration and makes comparisons within the same class possible. However, balanced coverage is not complete control of confounders: recordings follow a predefined sequence, and the paper does not establish randomized condition order.

Section 3.3 identifies the capture device as a head-mounted GoPro HERO13 recording indoor video at 3840×2160 and 30 fps. Full lighting activates two ceiling LED panels; low lighting deactivates one, with ambient daylight also mentioned in the introduction. “Low” therefore names a hardware configuration, not a guarantee that every point along the route is dark or outside the full-lighting illuminance range. The walking protocol in Section 3.3 uses an approximately 5 m path at roughly 1.2 m/s to introduce position-dependent lighting and natural movement disturbances. This is closer to mobile interaction than stationary capture, but remains a controlled indoor setting.

2. Spatial illuminance documentation: make environmental variation measurable

A calibrated illuminance meter records readings at approximately 80 spatial positions, with forward and backward measurements documenting variation along the space. These records describe environmental illumination rather than assigning only a bright/dim label to an entire video. A wearer can pass underneath a panel and then between panels without changing the lighting configuration, which explains why position-dependent measurements add information beyond scene labels. Section 3.3 gives ranges of 173–523 lux for full lighting and 29–460 lux for low lighting; their overlap shows that configuration labels are not interchangeable with actual lux readings.

Section 5.3 reports mean full-lighting readings of 310.0 lux forward and 326.4 lux backward, alongside coefficients of variation. A coefficient of variation expresses standard deviation relative to the mean, conventionally as a percentage; it describes relative measurement variability, not recognition accuracy. Stable local readings and substantial differences between spatial positions can coexist, but the main text does not fully specify the repeated-measurement and aggregation procedure behind these coefficients. Spatial documentation also does not establish precise per-frame illumination at the hands: the paper does not describe a frame-level position-to-lux alignment procedure. The metadata is therefore an experimental resource for illumination research, not an already completed causal analysis of lighting effects on each prediction.

3. Skeleton extraction and quality control: expose upstream tracking difficulty

MediaPipe Hands extracts 21 three-dimensional landmarks per hand, with frames below 0.5 detection confidence flagged for manual attention. Annotators inspect bounding boxes and use box-guided re-detection, followed by interpolation of missing landmarks and temporal smoothing. The authors inspect inter-annotator agreement on 10% of the data and report mean landmark deviation below 5 pixels; this measures annotation consistency, not error against independent three-dimensional ground truth. Per-gesture detection rates further reveal that upstream tracking difficulty varies systematically across classes. Simple single-hand poses are generally easier to detect than bilateral framing gestures, which suffer from mutual hand occlusion in the egocentric view.

The training description represents each frame with 42 landmarks for two hands and three coordinates per landmark, zero-padding the absent hand for single-hand gestures. The static release selects one representative frame per sequence, but the main text does not provide a sufficiently explicit rule for reproducing that selection. The dynamic release retains 30 fps sequences, exposing motion direction and observations before and after temporary occlusion. These versions support investigation of whether instantaneous shape is sufficient, but manual corrections, interpolation, and smoothing make released skeletons different from uncorrected live tracker outputs. Classification on cleaned skeletons should not be presented as the performance of an entirely automatic camera-to-label system under the same imaging difficulties.

4. Static–dynamic evaluation: separate recordings before extracting overlapping samples

The authors pool recordings per gesture across participants, lighting configurations, and mobility states, then use a fixed seed of 42 for an 80%/20% training–validation split. The important boundary is the sequence file rather than an independently sampled temporal window. Dynamic training uses 16-frame windows with stride 8, so adjacent windows share half their frames and could become near-duplicates across partitions if split independently. Keeping all windows from one recording in the same partition prevents this particular form of leakage. It does not prevent the same participant from appearing in both partitions, so the reported results concern unseen recordings rather than demonstrated generalization to unseen users.

Dynamic inputs append the coordinate displacement between a window's initial and middle frames to the original coordinates, expanding from three to six channels to help distinguish motion direction. ST-GCN treats landmarks as graph nodes and anatomical connections as edges, combining spatial graph and temporal convolutions to produce probabilities over 19 classes. E-ST-GCN adds joint reliability gating based on confidence and visibility, spatial attention over informative joint relationships, and temporal attention over salient motion. The main text explains these roles but does not provide enough computational detail to reconstruct the attention mechanisms, so no unsupported attention equations are supplied here. The static–dynamic comparison also changes preprocessing, displacement channels, and parameter counts; its gain is evidence for the overall dynamic setting, not a rigorously isolated temporal-information effect.

Loss & Training

Models train for 120 epochs with batch size 32, AdamW, and weighted cross-entropy over 19 classes. The class weight is total sample count divided by the product of the class count and that class's sample count, reducing the effect of uneven class frequencies. The extracted loss equation is damaged, but its accompanying prose supports this definition; the corrupted formula is not reconstructed. Section 4.1 describes nine spatial graph-convolution layers with three layers each at 64, 128, and 256 channels, plus temporal kernels of size 9. However, its 64-frame, 21-node, three-channel input and parameter count do not match the practical training description in Section 4.2 and Table 3; these specifications should not be silently combined. At inference, the benchmark predicts a gesture from supplied skeleton data, not from a demonstrated online chain including capture, pose estimation, and manual correction.

Key Experimental Results

Main Results

The following selected results come from Table 3 and use BiCE-HG's sequence-file-level 20% validation partition. Higher accuracy and F1 are better; the paper does not clearly identify the F1 averaging convention. These are separately trained dataset baselines, not cross-dataset SOTA comparisons.

Input release	Baseline setting	Accuracy (%) ↑	F1 ↑	Parameters
Static	No attention	81.45	0.804	214,291
Static	Reliability + spatial + temporal attention	80.38	0.796	263,913
Dynamic	No attention	88.57	0.878	214,387
Dynamic	Reliability + spatial + temporal attention	89.24	0.879	264,009

Calculated from Table 3, the no-attention dynamic setting improves on its static counterpart by 7.12 percentage points; full attention adds another 0.67 points in the dynamic setting. Attention reduces static accuracy by 1.07 points, so architectural complexity is not uniformly beneficial. The abstract and conclusion instead report 81.24% for static no-attention accuracy, conflicting with Table 3's 81.45%; this note consistently uses the table and does not repeat the paper's inconsistent temporal-gain claims.

Ablation Study

Table 3 compares full attention with no attention but does not separately remove the three attention components. The second table instead presents genuine data-quality analysis from Section 3.5 and Figure 3: MediaPipe detection success rates, where higher is better, rather than gesture classification accuracy. Its scope is all acquisition conditions, not the 20% classification validation partition, so the percentages in the two tables are not interchangeable metrics.

Gesture	Detection rate (%) ↑	Difficulty described in the paper
pinch	99.98	Relatively easy single-hand static pose
open palm	99.54	Relatively easy single-hand static pose
draw circle CW	95.52	Transient blur along a moving trajectory
hand frame right up	84.98	Egocentric inter-hand occlusion
hand frame left up	83.02	Egocentric inter-hand occlusion

Key Findings

• Important failures originate before classification: bilateral framing gestures have substantially lower tracking success, despite the downstream classifier operating only on skeletons.
• Temporal context helps without resolving every directional ambiguity: Section 5.2 reports only 50% and 18% dynamic classification accuracy for clockwise and counterclockwise circular drawing, respectively.
• The dynamic attention gain is small and lacks multi-seed uncertainty estimates; a 0.67-point difference should not be treated as a robustly significant architecture breakthrough.
• The 89.24% result pools lighting conditions. Without illuminance bins, cross-lighting train/test splits, or separate darkest-region results, it cannot establish that lighting does not affect recognition.

Highlights & Insights

• The main contribution is measurable environmental metadata rather than a generic recognizer presented as a new model. Lux and position records create a concrete entry point for condition-specific evaluation.
• Releasing both single-frame and temporal representations helps reveal which gestures require motion rather than shape alone. Such comparisons still need to control displacement channels and other preprocessing differences.
• Reporting tracking and classification separately helps identify the source of failure. For mutually occluding hands, improving landmarks may deserve testing before changing only the classifier.

Limitations & Future Work

• The authors explicitly acknowledge the lack of participant independence and leave subject-independent evaluation to future work; current accuracy does not characterize deployment to new users.
• Reader assessment: pooled recognition accuracy and lighting statistics are not the same as condition-resolved recognition analysis. Illumination robustness is a research direction enabled by the dataset, not a fully established performance conclusion.
• Dataset counts conflict: Section 3.2 mentions 2,622 raw sequences followed by 12,540 valid sequences after filtering without explaining the increase. This note uses the 2,622 instances consistently reported in Section 3.6 and the conclusion, without inferring an unreported window count.
• Capture and implementation specifications also conflict: Figure 1 differs from the prose on resolution and frame rate; the route is described as both 4 m and 5 m; node counts, window lengths, frame counts, and parameter counts vary. Reproduction requires checking the release and implementation rather than merging these descriptions.
• Illuminance ranges differ across sections, and 29–523 lux does not have a maximum-to-minimum ratio of 4.95. These statements are not treated as one consistent quantitative claim.
• Attention computations, representative-frame selection, window-to-recording prediction aggregation, and F1 averaging remain insufficiently specified. The cached text contains the full dataset method and experiments, but these details cannot be verified from it.
• Skeleton-classification speed on an RTX 3090 is not end-to-end speed on a headset. Automatic pose extraction, latency, and condition-specific testing would be more informative for deployment than emphasizing high FPS alone.

Related Work & Insights

• vs EgoGesture: EgoGesture covers more gesture categories and scenes; BiCE-HG distinguishes itself through measured illuminance and crossed mobility conditions, not demonstrated accuracy superiority over EgoGesture.
• vs FPHA / H2O: These datasets emphasize first-person hand actions or two-hand object interaction, whereas BiCE-HG targets interface gestures and lighting documentation. Their vocabularies and evaluation goals differ.
• vs standard ST-GCN baselines: Graph-temporal skeleton models establish starting results here, with tri-attention as a baseline enhancement. The reusable contribution remains the condition-documented data.

Rating

• Novelty: 3/5. Quantified illumination combined with mobility conditions is useful, while the recognition architecture is not the central innovation.
• Experimental Thoroughness: 2/5. Static, dynamic, and tracking-quality analyses are present, but participant-independent and illuminance-resolved tests are missing and numerical inconsistencies remain.
• Writing Quality: 2/5. The dataset motivation is clear, but conflicting capture settings, counts, and results weaken reproducibility.
• Value: 3/5. A useful starting point for controlled indoor illumination studies, provided the split and actual release are inspected before use.