Skip to main content
  • Home
  • Happenings
  • Events
  • Realistic 3D Hand Tracking
Realistic 3D Hand Tracking

Realistic 3D Hand Tracking

Date20th Jul 2021

Time04:00 PM

Venue Google Meet (see link).

PAST EVENT

Details

Depth-based 3D hand trackers are expected to estimate highly accurate poses of the human hand given the image. One of the critical problems in tracking the hand pose is the generation of realistic predictions. This seminar proposes a novel "anatomical filter" that accepts a hand pose from a hand tracker and generates the closest possible pose within the real human hand's anatomical bounds. The filter works by calculating the 26-DoF vector representing the joint angles and correcting those angles based on the real human hand's biomechanical limitations. The proposed filter can be plugged into any hand tracker to enhance its performance. The filter has been tested on two state-of-the-art 3D hand trackers. The empirical observations show that our proposed filter improves the hand pose's anatomical correctness and allows a smooth trade-off with pose error. The filter achieves the lowest prediction error when used with state-of-the-art trackers at 10% correction. Using this concept, we also propose the Single Shot Corrective CNN (SSC-CNN) framework to tackle the problem at the architecture level. In contrast to previous works which use post-facto pose filters, SSC-CNN predicts the hand pose that implicitly conforms to the human hand's biomechanical bounds and rules in a single forward pass. The model was trained and tested on the HANDS2017 and MSRA datasets. Experiments show that our proposed model shows comparable accuracy to the state-of-the-art models. However, the previous methods have high anatomical errors whereas our model is free from such errors. Experiments also show that the ground truth provided in the datasets used also suffer from anatomical errors and an Anatomical Error Free (AEF) version of the datasets namely AEF-HANDS2017 and AEF-MSRA was created. Future works include incorporating biomechanically constrained velocity bounds in the network architecture.

Speakers

Joseph H. R. Isaac

Computer Science and Engg.