Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learning

Mar 10, 2021
Dilip Arumugam, Peter Henderson, Pierre-Luc Bacon

How do we formalize the challenge of credit assignment in reinforcement learning? Common intuition would draw attention to reward sparsity as a key contributor to difficult credit assignment and traditional heuristics would look to temporal recency for the solution, calling upon the classic eligibility trace. We posit that it is not the sparsity of the reward itself that causes difficulty in credit assignment, but rather the \emph{information sparsity}. We propose to use information theory to define this notion, which we then use to characterize when credit assignment is an obstacle to efficient learning. With this perspective, we outline several information-theoretic mechanisms for measuring credit under a fixed behavior policy, highlighting the potential of information theory as a key tool towards provably-efficient credit assignment.

* Workshop on Biological and Artificial Reinforcement Learning (NeurIPS 2020) 

Share this with someone who'll enjoy it:

   Access Paper Source

Share this with someone who'll enjoy it: