Evaluation Report

HA-HAND

Sub-centimeter absolute 3D hand tracking

Ishan Ahluwalia · Samay Maini · Rushil Agarwal
Human Archive Research · August 2026

How we evaluated

Ground truth is marker-based, captured on hardware we built ourselves, and scored with no alignment of any kind. The rig is a compact global-shutter stereo camera (60 mm baseline, head-mountable) tracked as a rigid body inside an OptiTrack motion capture volume running at 240 Hz, hardware-genlocked to the camera shutter. Retroreflective markers on the hand are tracked at sub-millimeter accuracy, and a parametric hand model is fit through the marker trajectories to recover anatomical joint centers — the same class of methodology used by Meta's HOT3D benchmark. The camera's mocap pose gives a per-frame hand-eye transform into the camera frame, verified by reprojection; residual time offset between the two systems was measured at under a third of a frame.

The evaluation capture session comprises 2,109 frames of naturalistic hand motion, and all metrics below are reported over the full clip.

We report two metrics:

Absolute MPJPE: Mean per-joint position error in the camera frame, with no alignment of any kind. This is the primary metric: it is the error a downstream consumer of the tracking actually experiences.
PA-MPJPE: Per-joint error after full Procrustes alignment (translation, rotation, and scale) — measures hand shape and articulation only. This is the most commonly reported metric in the literature.

Results

Accuracy over the full 2,109-frame clip (21 joints per frame). All values in millimeters, absolute (unaligned) unless stated.
Metric	Value
Absolute MPJPE (mean)	8.79 mm
Absolute MPJPE (median)	6.66 mm
PA-MPJPE	4.98 mm

The full evaluation take is shown below, with prediction and ground truth rendered in absolute camera coordinates.

The full evaluation take. Green: mocap-derived ground-truth skeleton. Orange: HA-HAND prediction, in absolute camera coordinates — no alignment applied.

The same take in the motion-capture world frame: head pose (camera frustum and trajectory) with the green ground-truth and orange predicted hand skeletons in absolute 3D space.

The approach, briefly

HA-HAND fuses three sources of evidence, none sufficient alone:

A hand-mesh prior from a fine-tuned egocentric hand-pose capture model, giving a strong articulated estimate of hand shape and pose in every frame.
Learned dense stereo computed over the hand surface, recovering metric-scale geometry directly from the calibrated stereo pair instead of leaning on monocular depth priors.
Two-view and temporal consistency constraints that force every prediction to agree with both cameras and with its neighbors in time.

A per-frame refinement stage treats the predicted hand as a differentiable object and optimizes it until it satisfies all three sources at once.

What’s next?

Soon, we will begin collecting data on our ground-truth motion capture setup and scaling that collection across diverse real-world environments to post-train our annotation models. We are also extending our evaluation to cross-session captures and public benchmarks such as HOT3D, while continuing to scale large-scale egocentric manipulation data collection for robot learning.

If you’re interested in working with us, please write to raj@humanarchive.ai.