Your smartwatch may be more powerful than you think. Researchers at Cornell University and KAIST in South Korea have developed a system called WatchHand that turns any regular smartwatch into a hand tracking device.
The best part is that no additional hardware is required. According to the published article, the technology uses AI-powered microsonar to read your hand movements and finger positions in real time, and works with hardware your watch already has.
How does WatchHand actually work?
Your smartwatch has a built-in speaker and microphone. WatchHand does both in ways you would never expect. The speaker emits inaudible sound waves that reflect from your hand back into the microphone, creating a unique echo pattern.
An AI algorithm running directly on the watch reads this pattern and maps your hand position in 3D. Think of it like the echolocation that bats use to navigate, only it fits on your wrist. Since everything is processed on the watch itself, your movement data is not sent anywhere else.
What can WatchHand do for you?
The practical applications are really exciting. A title could be skipped by double-tapping with your fingers. Subtle hand gestures let you move your cursor without touching your mouse or trackpad.
In the long term, WatchHand could potentially track your input, serve as a controller in augmented or virtual reality, and support people with limited mobility who have difficulty using traditional input devices.
Researchers also believe that a software update alone could bring this capability to millions of existing smartwatches already in people’s hands.
However, there are some limitations worth knowing. WatchHand currently only works on Android devices, not Apple Watches, and struggles to keep up when walking. But for a unique system based entirely on hardware you already own, that’s a pretty good start.
Smartwatches are clearly becoming a lot more sophisticated than expected, with researchers also working on a wrist-worn device that can detect microplastics in your body.
