Decision Support for Video-based Detection of Flu Symptoms

The development of decision support systems is a growing domain that can be applied in the area of disease control and diagnostics. Using video-based surveillance data, skeleton features are extracted to perform action recognition, specifically the detection and recognition of coughing and sneezing motions. Providing evidence of flu-like symptoms, a decision support system based on causal networks is capable of providing the operator with vital information for decision-making. A modified residual temporal convolutional network is proposed for action recognition using skeleton features. This paper addresses the capability of using results from a machine-learning model as evidence for a cognitive decision support system. We propose risk and trust measures as a metric to bridge between machine-learning and machine-reasoning. We provide experiments on evaluating the performance of the proposed network and how these performance measures can be combined with risk to generate trust.

Reliability of Decision Support in Cross-spectral Biometric-enabled Systems

This paper addresses the evaluation of the performance of the decision support system that utilizes face and facial expression biometrics. The evaluation criteria include risk of error and related reliability of decision, as well as their contribution to the changes in the perceived operator's trust in the decision. The relevant applications include human behavior monitoring and stress detection in individuals and teams, and in situational awareness system. Using an available database of cross-spectral videos of faces and facial expressions, we conducted a series of experiments that demonstrate the phenomenon of biases in biometrics that affect the evaluated measures of the performance in human-machine systems.

Assessing Risks of Biases in Cognitive Decision Support Systems

Recognizing, assessing, countering, and mitigating the biases of different nature from heterogeneous sources is a critical problem in designing a cognitive Decision Support System (DSS). An example of such a system is a cognitive biometric-enabled security checkpoint. Biased algorithms affect the decision-making process in an unpredictable way, e.g. face recognition for different demographic groups may severely impact the risk assessment at a checkpoint. This paper addresses a challenging research question on how to manage an ensemble of biases? We provide performance projections of the DSS operational landscape in terms of biases. A probabilistic reasoning technique is used for assessment of the risk of such biases. We also provide a motivational experiment using face biometric component of the checkpoint system which highlights the discovery of an ensemble of biases and the techniques to assess their risks.

Information Fusion on Belief Networks

This paper will focus on the process of 'fusing' several observations or models of uncertainty into a single resultant model. Many existing approaches to fusion use subjective quantities such as 'strengths of belief' and process these quantities with heuristic algorithms. This paper argues in favor of quantities that can be objectively measured, as opposed to the subjective 'strength of belief' values. This paper will focus on probability distributions, and more importantly, structures that denote sets of probability distributions known as 'credal sets'. The novel aspect of this paper will be a taxonomy of models of fusion that use specific types of credal sets, namely probability interval distributions and Dempster-Shafer models. An objective requirement for information fusion algorithms is provided, and is satisfied by all models of fusion presented in this paper. Dempster's rule of combination is shown to not satisfy this requirement. This paper will also assess the computational challenges involved for the proposed fusion approaches.

CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

Human activity and gesture recognition is an important component of rapidly growing domain of ambient intelligence, in particular in assisting living and smart homes. In this paper, we propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN), for automated hand gesture recognition using both depth and skeleton data. Each of these types of data can be used separately to train neural networks to recognize hand gestures. While RNN were reported previously to perform well in recognition of sequences of movement for each skeleton joint given the skeleton information only, this study aims at utilizing depth data and apply CNN to extract important spatial information from the depth images. Together, the tandem CNN+RNN is capable of recognizing a sequence of gestures more accurately. As well, various types of fusion are studied to combine both the skeleton and depth information in order to extract temporal-spatial information. An overall accuracy of 85.46% is achieved on the dynamic hand gesture-14/28 dataset.

Risk Assessment in the Face-based Watchlist Screening in e-Border

This paper concerns with facial-based watchlist technology as a component of automated border control machines deployed in e-borders. The key task of the watchlist technology is to mitigate effects of mis-identification and impersonation. To address this problem, we developed a novel cost-based model of traveler risk assessment and proved its efficiency via intensive experiments using large-scale facial databases. The results of this study are applicable to any biometric modality to be used in watchlist technology.

Multi-Metric Evaluation of Thermal-to-Visual Face Recognition

In this paper, we aim to address the problem of heterogeneous or cross-spectral face recognition using machine learning to synthesize visual spectrum face from infrared images. The synthesis of visual-band face images allows for more optimal extraction of facial features to be used for face identification and/or verification. We explore the ability to use Generative Adversarial Networks (GANs) for face image synthesis, and examine the performance of these images using pre-trained Convolutional Neural Networks (CNNs). The features extracted using CNNs are applied in face identification and verification. We explore the performance in terms of acceptance rate when using various similarity measures for face verification.