Continuous sign language recognition from wearable IMUs using deep capsule networks and game theory

Sign Language is used by the deaf community all over world. The work presented here proposes a novel one-dimensional deep capsule network (CapsNet) architecture for continuous Indian Sign Language recognition by means of signals obtained from a custom designed wearable IMU system. The performance of the proposed CapsNet architecture is assessed by altering dynamic routing between capsule layers. The proposed CapsNet yields improved accuracy values of 94% for 3 routings and 92.50% for 5 routings in comparison with the convolutional neural network (CNN) that yields an accuracy of 87.99%. Improved learning of the proposed architecture is also validated by spatial activations depicting excited units at the predictive layer. Finally, a novel non-cooperative pick-and-predict competition is designed between CapsNet and CNN. Higher value of Nash equilibrium for CapsNet as compared to CNN indicates the suitability of the proposed approach.

Transfer Learning for sEMG-based Hand Gesture Classification using Deep Learning in a Master-Slave Architecture

Recent advancements in diagnostic learning and development of gesture-based human machine interfaces have driven surface electromyography (sEMG) towards significant importance. Analysis of hand gestures requires an accurate assessment of sEMG signals. The proposed work presents a novel sequential master-slave architecture consisting of deep neural networks (DNNs) for classification of signs from the Indian sign language using signals recorded from multiple sEMG channels. The performance of the master-slave network is augmented by leveraging additional synthetic feature data generated by long short term memory networks. Performance of the proposed network is compared to that of a conventional DNN prior to and after the addition of synthetic data. Up to 14% improvement is observed in the conventional DNN and up to 9% improvement in master-slave network on addition of synthetic data with an average accuracy value of 93.5% asserting the suitability of the proposed approach.

Activity Detection from Wearable Electromyogram Sensors using Hidden Markov Model

Surface electromyography (sEMG) has gained significant importance during recent advancements in consumer electronics for healthcare systems, gesture analysis and recognition and sign language communication. For such a system, it is imperative to determine the regions of activity in a continuously recorded sEMG signal. The proposed work provides a novel activity detection approach based on Hidden Markov Models (HMM) using sEMG signals recorded when various hand gestures are performed. Detection procedure is designed based on a probabilistic outlook by making use of mathematical models. The requirement of a threshold for activity detection is obviated making it subject and activity independent. Correctness of the predicted outputs is asserted by classifying the signal segments around the detected transition regions as activity or rest. Classified outputs are compared with the transition regions in a stimulus given to the subject to perform the activity. The activity onsets are detected with an average of 96.25% accuracy whereas the activity termination regions with an average of 87.5% accuracy with the considered set of six activities and four subjects.

Dual Stage Classification of Hand Gestures using Surface Electromyogram

Surface electromyography (sEMG) is becoming exceeding useful in applications involving analysis of human motion such as in human-machine interface, assistive technology, healthcare and prosthetic development. The proposed work presents a novel dual stage classification approach for classification of grasping gestures from sEMG signals. A statistical assessment of these activities is presented to determine the similar characteristics between the considered activities. Similar activities are grouped together. In the first stage of classification, an activity is identified as belonging to a group, which is then further classified as one of the activities within the group in the second stage of classification. The performance of the proposed approach is compared to the conventional single stage classification approach in terms of classification accuracies. The classification accuracies obtained using the proposed dual stage classification are significantly higher as compared to that for single stage classification.

Classification of Hand Gestures from Wearable IMUs using Deep Neural Network

IMUs are gaining significant importance in the field of hand gesture analysis, trajectory detection and kinematic functional study. An Inertial Measurement Unit (IMU) consists of tri-axial accelerometers and gyroscopes which can together be used for formation analysis. The paper presents a novel classification approach using a Deep Neural Network (DNN) for classifying hand gestures obtained from wearable IMU sensors. An optimization objective is set for the classifier in order to reduce correlation between the activities and fit the signal-set with best performance parameters. Training of the network is carried out by feed-forward computation of the input features followed by the back-propagation of errors. The predicted outputs are analyzed in the form of classification accuracies which are then compared to the conventional classification schemes of SVM and kNN. A 3-5% improvement in accuracies is observed in the case of DNN classification. Results are presented for the recorded accelerometer and gyroscope signals and the considered classification schemes.

Convolutional Neural Network Array for Sign Language Recognition using Wearable IMUs

Advancements in gesture recognition algorithms have led to a significant growth in sign language translation. By making use of efficient intelligent models, signs can be recognized with precision. The proposed work presents a novel one-dimensional Convolutional Neural Network (CNN) array architecture for recognition of signs from the Indian sign language using signals recorded from a custom designed wearable IMU device. The IMU device makes use of tri-axial accelerometer and gyroscope. The signals recorded using the IMU device are segregated on the basis of their context, such as whether they correspond to signing for a general sentence or an interrogative sentence. The array comprises of two individual CNNs, one classifying the general sentences and the other classifying the interrogative sentence. Performances of individual CNNs in the array architecture are compared to that of a conventional CNN classifying the unsegregated dataset. Peak classification accuracies of 94.20% for general sentences and 95.00% for interrogative sentences achieved with the proposed CNN array in comparison to 93.50% for conventional CNN assert the suitability of the proposed approach.