Researchers Create A Wrist-Mounted Device That Can Continuously Track The Entire Hand

Researchers from Cornell and the University of Wisconsin, Madison, have announced the development of a new wrist-mounted device that's able to track the entire human hand in 3D continuously. The device is called FingerTrak and translates the many positions of the human hand into 3D information, including information on 20 finger joint positions.FingerTrak uses three or four miniature, low-resolution thermal cameras that read the contours on the wrist. Researchers say that the device can be used to translate sign language or used in virtual-reality. It can also be used for mobile health, human-robot interaction, and in other areas.

Scientists on the project say that the major discovery by the team was that by looking at the wrist contours, the technology was able to reconstruct in 3D with high accuracy where the fingers are. This is the first system able to reconstruct full hand posture based on the contours of the wrist.

In the past, wrist-mounted cameras have been too bulky and obtrusive for everyday use. Despite the bulkiness, most were only able to reconstruct a few discrete hand gestures. The breakthrough FingerTrak system is a lightweight bracelet that allows for free hand movement. The device focuses on a combination of thermal imaging and machine learning to virtually reconstruct hand movements rather than relying on cameras to capture the position of the fingers directly.

The four tiny thermal cameras used on the device are each about the size of a pea and can snap multiple silhouette images to form an outline of the hand. The system can combine the images and reconstruct the virtual hand in 3D using a deep neural network. The technology allows the capture of the entire hand pose even if the hand is holding an object.