Alert button

Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture

Sep 21, 2019
Pawel Ladosz, Eseoghene Ben-Iwhiwhu, Yang Hu, Nicholas Ketz, Soheil Kolouri, Jeffrey L. Krichmar, Praveen Pilly, Andrea Soltoggio

Figure 1 for Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture
Figure 2 for Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture
Figure 3 for Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture
Figure 4 for Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture

Share this with someone who'll enjoy it:

This paper introduces the modulated Hebbian plus Q network architecture (MOHQA) for solving challenging partially observable Markov decision processes (POMDPs) deep reinforcement learning problems with sparse rewards and confounding observations. The proposed architecture combines a deep Q-network (DQN), and a modulated Hebbian network with neural eligibility traces (MOHN). Bio-inspired neural traces are used to bridge temporal delays between actions and rewards. The purpose is to discover distal cause-effect relationships where confounding observations and sparse rewards cause standard RL algorithms to fail. Each of the two modules of the network (DQN and MOHN) is responsible for different aspects of learning. DQN learns low level features and control, while MOHN contributes to the high-level decisions by bridging rewards with past actions. The strength of the approach is to support a DQN standard framework when temporal difference errors are difficult to compute due to non-observable states. The system is tested on a set of generalized decision making problems encoded as decision tree graphs that deliver delayed rewards after key decision points and confounding observations. The simulations show that the proposed approach helps solve problems that are currently challenging for state-of-the-art deep reinforcement learning algorithms.

View paper onarxiv icon

Share this with someone who'll enjoy it: