Black box meta-learning intrinsic rewards for sparse-reward environments

Despite the successes and progress of deep reinforcement learning over the last decade, several challenges remain that hinder its broader application. Some fundamental aspects to improve include data efficiency, generalization capability, and ability to learn in sparse-reward environments, which often require human-designed dense rewards. Meta-learning has emerged as a promising approach to address these issues by optimizing components of the learning algorithm to meet desired characteristics. Additionally, a different line of work has extensively studied the use of intrinsic rewards to enhance the exploration capabilities of algorithms. This work investigates how meta-learning can improve the training signal received by RL agents. The focus is on meta-learning intrinsic rewards under a framework that doesn't rely on the use of meta-gradients. We analyze and compare this approach to the use of extrinsic rewards and a meta-learned advantage function. The developed algorithms are evaluated on distributions of continuous control tasks with both parametric and non-parametric variations, and with only sparse rewards accessible for the evaluation tasks.

A Strong Inductive Bias: Gzip for binary image classification

Deep learning networks have become the de-facto standard in Computer Vision for industry and research. However, recent developments in their cousin, Natural Language Processing (NLP), have shown that there are areas where parameter-less models with strong inductive biases can serve as computationally cheaper and simpler alternatives. We propose such a model for binary image classification: a nearest neighbor classifier combined with a general purpose compressor like Gzip. We test and compare it against popular deep learning networks like Resnet, EfficientNet and Mobilenet and show that it achieves better accuracy and utilizes significantly less space, more than two order of magnitude, within a few-shot setting. As a result, we believe that this underlines the untapped potential of models with stronger inductive biases in few-shot scenarios.

Equilivest: A Robotic Vest to aid in Post-Stroke Dynamic Balance Rehabilitation

Stroke is a medical condition that can affect motor function, particularly dynamic balance. Biofeedback can aid in rehabilitation procedures which help patients to regain lost motor activity and recover functionality. In this work, we are presenting a robotic smart-vest device that can analyze Inertial Measurement Unit (IMU) data and assist in rehabilitation procedures by providing timed feedback in the form of vibrotactile stimulation. Information provided by principal caregivers and patients in the form of surveys and interviews, is used to hypothesize potential clinical causes and to derive alternative three alternative clinical modalities: Artificial Vestibular Feedback, Gait Pacemaker and Risk-Predictor.

Report: EPOC Emotiv EEG Basics

This document provides some basic guidance to start working with the EPOC Emotiv neuroheadset device and describes how to use it to perform basic Brain-Computer Interface (BCI) research. A brief tutorial on how to set up the device, from its electrophysiological point of view, as well as a description and practical code to perform some basic analysis, is explained. A basic experiment is introduced to detect one of the oldest and, indeed, quite still valuable electrophysiological correlate, visual occipital alpha waves, or Berger Rhythm. An additional experiment is expounded where the power spectrum of alpha waves is reduced when a subject is affected by background cognitive disturbances. This document also briefs about the extraction of information by using the EPOC Emotiv library and also with python Emokit package. This report presents a basic guide on how to use EEGLAB and MATLAB, as well as python stack to perform the neurophysiological analysis. Finally, a basic analysis on different feature extraction and classification methods is provided.