mmWrite: Passive Handwriting Tracking Using a Single Millimeter-Wave Radio

In the era of pervasively connected and sensed Internet of Things, many of our interactions with machines have been shifted from conventional computer keyboards and mouses to hand gestures and writing in the air. While gesture recognition and handwriting recognition have been well studied, many new methods are being investigated to enable pervasive handwriting tracking. Most of the existing handwriting tracking systems either require cameras and handheld sensors or involve dedicated hardware restricting user convenience and the scale of usage. In this article, we present <italic>mmWrite</italic>, the first high-precision passive handwriting tracking system using a single commodity millimeter-wave (mmWave) radio. Leveraging the short wavelength and large bandwidth of 60-GHz signals and the radar-like capabilities enabled by the large phased array, <italic>mmWrite</italic> transforms any flat region into an interactive writing surface that supports handwriting tracking at millimeter accuracy. <italic>MmWrite</italic> employs an end-to-end pipeline of signal processing to enhance the range and spatial resolution limited by the hardware, boost the coverage, and suppress interference from backgrounds and irrelevant objects. We implement and evaluate <italic>mmWrite</italic> on a commodity 60-GHz device. The experimental results show that <italic>mmWrite</italic> can track a finger/pen with a median error of 2.8 mm and thus can reproduce handwritten characters as small as 1 cm <inline-formula> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 1 cm, with a coverage of up to 8 m² supported. With minimal infrastructure needed, <italic>mmWrite</italic> promises ubiquitous handwriting tracking for new applications in the field of human–computer interactions.