Improving Environment Robustness of Deep Reinforcement Learning Approaches for Autonomous Racing Using Bayesian Optimization-based Curriculum Learning

Deep reinforcement learning (RL) approaches have been broadly applied to a large number of robotics tasks, such as robot manipulation and autonomous driving. However, an open problem in deep RL is learning policies that are robust to variations in the environment, which is an important condition for such systems to be deployed into real-world, unstructured settings. Curriculum learning is one approach that has been applied to improve generalization performance in both supervised and reinforcement learning domains, but selecting the appropriate curriculum to achieve robustness can be a user-intensive process. In our work, we show that performing probabilistic inference of the underlying curriculum-reward function using Bayesian Optimization can be a promising technique for finding a robust curriculum. We demonstrate that a curriculum found with Bayesian optimization can outperform a vanilla deep RL agent and a hand-engineered curriculum in the domain of autonomous racing with obstacle avoidance. Our code is available at https://github.com/PRISHIta123/Curriculum_RL_for_Driving.

A Novel U-Net Architecture for Denoising of Real-world Noise Corrupted Phonocardiogram Signal

The bio-acoustic information contained within heart sound signals are utilized by physicians world-wide for auscultation purpose. However, the heart sounds are inherently susceptible to noise contamination. Various sources of noises like lung sound, coughing, sneezing, and other background noises are involved in such contamination. Such corruption of the heart sound signal often leads to inconclusive or false diagnosis. To address this issue, we have proposed a novel U-Net based deep neural network architecture for denoising of phonocardiogram (PCG) signal in this paper. For the design, development and validation of the proposed architecture, a novel approach of synthesizing real-world noise corrupted PCG signals have been proposed. For the purpose, an open-access real-world noise sample dataset and an open-access PCG dataset has been utilized. The performance of the proposed denoising methodology has been evaluated on the synthesized noisy PCG dataset. The performance of the proposed algorithm has been compared with existing state-of-the-art (SoA) denoising algorithms qualitatively and quantitatively. The proposed denoising technique has shown improvement in performance as comparison to the SoAs.

Murmur Detection Using Parallel Recurrent & Convolutional Neural Networks

In this article, we propose a novel technique for classification of the Murmurs in heart sound. We introduce a novel deep neural network architecture using parallel combination of the Recurrent Neural Network (RNN) based Bidirectional Long Short-Term Memory (BiLSTM) & Convolutional Neural Network (CNN) to learn visual and time-dependent characteristics of Murmur in PCG waveform. Set of acoustic features are presented to our proposed deep neural network to discriminate between Normal and Murmur class. The proposed method was evaluated on a large dataset using 5-fold cross-validation, resulting in a sensitivity and specificity of 96 +- 0.6 % , 100 +- 0 % respectively and F1 Score of 98 +- 0.3 %.

Towards Wide Learning: Experiments in Healthcare

In this paper, a Wide Learning architecture is proposed that attempts to automate the feature engineering portion of the machine learning (ML) pipeline. Feature engineering is widely considered as the most time consuming and expert knowledge demanding portion of any ML task. The proposed feature recommendation approach is tested on 3 healthcare datasets: a) PhysioNet Challenge 2016 dataset of phonocardiogram (PCG) signals, b) MIMIC II blood pressure classification dataset of photoplethysmogram (PPG) signals and c) an emotion classification dataset of PPG signals. While the proposed method beats the state of the art techniques for 2nd and 3rd dataset, it reaches 94.38% of the accuracy level of the winner of PhysioNet Challenge 2016. In all cases, the effort to reach a satisfactory performance was drastically less (a few days) than manual feature engineering.