YoHa

A practical hand tracking engine.

Quick Links:

• Demo (Code)
• Docs

Installation

npm install @handtracking.io/yoha

Please note:

• You need to serve the files from node_modules/@handtracking.io/yoha. (Webpack Example)
• You need to serve your page with https. (Webpack Example)
• You should use cross-origin isolation as it improves the engine's performance in certain scenarios. (Webpack Example)

Description

YoHa is a hand tracking engine that is built with the goal of being strong in practical scenarios where hand tracking is employed to add true value to an application. While ultimately the goal is to be a general purpose hand tracking engine supporting any hand pose, the engine is built and will grow around specific hand poses that users/developers find useful. These poses are detected by the engine which allows to build applications with novel interactions. See the demo for an example.

YoHa is currently only available for the web.

YoHa is currently in beta.

About the name: YoHa is short for ("Your Hand Tracking").

Supported Poses

Currently YoHa supports and detects the following poses:

• Pinch
• Fist

Your help is needed to expand this list. Please open an issue or vote on an existing one to communicate which poses you would like to see next.

Technical Details

YoHa was build from scratch. It uses a custom neural network trained using a custom dataset. The backbone for the inference in the browser is currently TensorFlow.js as it currently provides the fastest open source inference for the web. However, this may change in the future depending on how the ML landscape for the web develops.

Features:

• Detection of 21 2D-landmark coordinates (single hand).
• Hand presence detection.
• Hand orientation (left/right hand) detection.
• Inbuilt pose detection.

Performance

YoHa was build with performance in mind. It is able to provide realtime user experience on a broad range of laptops and desktop devices. The performance on mobile devices is not great which hopefuly will change with the further development of inference frameworks like TensorFlow.js

Please note that native inference speed can not be compared with the web inference speed. Differently put, if you were to run YoHa natively it would be much faster than via the web browser.

Alternatives

The most prominent hand tracking solution for the web is from mediapipe. It is a very general and performant solution that keeps evolving and supports a lot of different deployment methods. In terms of performance the solution from mediapipe is faster. There is reason to believe this is to an unknown and hard-to-assess extend due to the inference engine that it uses which is closed source.

About

Hey, I'm Benjamin. I started out making this project because I wanted to make the web more interactive (especially due to covid imposed home office). Existing solutions did not offer what I was looking for so I built my own.

[email protected]