Overview
Use a depth camera to reconstruct the entire forearm and hand in real time, detect keypoints, and perform rigging/skin-ning. In AR, the user’s real arm motion drives the virtually reconstructed arm (including the hand) to perform the same movements, enabling synchronized visualization and interaction.
Scope & Targets
- Cover 5–8 subjects/scenes for end-to-end demonstrations.
- Per session (capture → reconstruction → rigging → AR preview/export) ≤ 2–3 minutes.
- Export true-scale 3D models of the forearm and hand (optional textures/UV/normals, with skeletal animation).
Core Method
- Acquisition & Pose: Depth + RGB; device calibration for camera poses.
- Full-Arm Reconstruction: TSDF/point-cloud fusion → meshing and light cleanup (optional texturing).
- Keypoints & Driving: Detect full-arm keypoints (wrist, elbow, finger joints, etc.) with temporal smoothing; 2D→3D back-projection or direct 3D.
- Rigging & AR Coupling: Bind the skeleton to the reconstructed full-arm mesh; in AR, use the live skeleton to drive the virtual reconstructed arm; one-click export.
Application
- Rehabilitation Assessment & Training: Real-time AR visualization of patient-specific arm motion with quantitative ROM, symmetry, and repeatability metrics, providing immediate visual feedback for elderly and post-operative upper-limb training