Pyro: Thumb-Tip Gesture Recognition Using Pyroelectric Infrared Sensing

Motivation

Micro finger gestures offer new opportunities for natural, subtle, fast, and unobtrusive interactions in wearable, mobile, and ubiquitous computing applications. For example, gesturing the thumb tip against the tip of the index finger is a natural method of performing input, requiring little effort from users because the index finger serves as a supporting surface to naturally provide haptic feedback. This motion introduces less fatigue over time compared with traditional gestural input methods, which often require moving the finger, hand, or even the entire arm in mid-air. However, tracking fine-grained thumb-tip gestures remains very challenging due to the small magnitude of finger motions and frequent occurrences of self-occlusion.

Prototype

We built our customized sensing board around a Cortex M4 micro-controller (MK20DX256VLH7) running at 96MHz, powered by the Teensy 3.2 firmware. The board has an LM324 based ADC preamp, a power management circuit, and a Bluetooth module. To reduce the dominant noise (50 kHz - 300 kHz) caused by powerline and fluorescent light ballasts, we implemented a bandpass filter with cut-off frequencies of 1.59 Hz and 486.75 Hz. The relatively wide bandwidth gives us the flexibility to explore sampling rates. After the noise is removed, the input signal is amplified with a gain of 33 and biased by AREF/2 (1.5 V) to preserve the fidelity of the analog signal. The gain value is carefully tuned to have an optimal sensing range of approximately 0.5 cm to 30 cm away from the PIR sensor. This design mitigates the background thermal infrared signals from the human body minimizing the impact on the foreground finger gesture signal.

Signal Processing

We collected raw gesturing data from ADC in microcontroller and extracted frequency domain and time domain features from them. We then fed the extracted features into RandomForest classifier getting the classification result. Here are typical samples of six thumb-tip micro gestures.

Six Thumb-tip Gestures	Signal Processing

Selected Press Coverage

EurekAlert && ACM TechNews SIGCHI Edition (November 2017): Dartmouth to debut wearables that warn and wow at UIST 2017

The Dartmouth: Dartmouth team visits tech symposium