Low-Power On-Device Gesture Recognition with Einsum Networks

We design a gesture-recognition pipeline for networks of distributed, resource constrained devices utilising Einsum Networks. Einsum Networks are probabilistic circuits that feature a tractable inference, explainability, and energy efficiency. The system is validated in a scenario of low-power, body-worn, passive Radio Frequency Identification-based gesture recognition. Each constrained device includes task-specific processing units responsible for Received Signal Strength (RSS) and phase processing or Angle of Arrival (AoA) estimation, along with feature extraction, as well as dedicated Einsum hardware that processes the extracted features. The output of all constrained devices is then fused in a decision aggregation module to predict gestures. Experimental results demonstrate that the method outperforms the benchmark models.

Angle of Arrival Estimation for Gesture Recognition from reflective body-worn tags

We investigate hand gesture recognition by leveraging passive reflective tags worn on the body. Considering a large set of gestures, distinct patterns are difficult to be captured by learning algorithms using backscattered received signal strength (RSS) and phase signals. This is because these features often exhibit similarities across signals from different gestures. To address this limitation, we explore the estimation of Angle of Arrival (AoA) as a distinguishing feature, since AoA characteristically varies during body motion. To ensure reliable estimation in our system, which employs Smart Antenna Switching (SAS), we first validate AoA estimation using the Multiple SIgnal Classification (MUSIC) algorithm while the tags are fixed at specific angles. Building on this, we propose an AoA tracking method based on Kalman smoothing. Our analysis demonstrates that, while RSS and phase alone are insufficient for distinguishing certain gesture data, AoA tracking can effectively differentiate them. To evaluate the effectiveness of AoA tracking, we implement gesture recognition system benchmarks and show that incorporating AoA features significantly boosts their performance. Improvements of up to 15% confirm the value of AoA-based enhancement.

Gesture Recognition from body-Worn RFID under Missing Data

We explore hand-gesture recognition through the use of passive body-worn reflective tags. A data processing pipeline is proposed to address the issue of missing data. Specifically, missing information is recovered through linear and exponential interpolation and extrapolation. Furthermore, imputation and proximity-based inference are employed. We represent tags as nodes in a temporal graph, with edges formed based on correlations between received signal strength (RSS) and phase values across successive timestamps, and we train a graph-based convolutional neural network that exploits graph-based self-attention. The system outperforms state-of-the-art methods with an accuracy of 98.13% for the recognition of 21 gestures. We achieve 89.28% accuracy under leave-one-person-out cross-validation. We further investigate the contribution of various body locations on the recognition accuracy. Removing tags from the arms reduces accuracy by more than 10%, while removing the wrist tag only reduces accuracy by around 2%. Therefore, tag placements on the arms are more expressive for gesture recognition than on the wrist.