Alert button

Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments

Add code
Alert button
Jul 14, 2021
Figure 1 for Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments
Figure 2 for Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments
Figure 3 for Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments
Figure 4 for Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments

Share this with someone who'll enjoy it:

Reinforcement Learning (RL) is known to be often unsuccessful in environments with sparse extrinsic rewards. A possible countermeasure is to endow RL agents with an intrinsic reward function, or 'intrinsic motivation', which rewards the agent based on certain features of the current sensor state. An intrinsic reward function based on the principle of empowerment assigns rewards proportional to the amount of control the agent has over its own sensors. We implemented a variation on a recently proposed intrinsically motivated agent, which we refer to as the 'curious' agent, and an empowerment-inspired agent. The former leverages sensor state encoding with a variational autoencoder, while the latter predicts the next sensor state via a variational information bottleneck. We compared the performance of both agents to that of an advantage actor-critic baseline in four sparse reward grid worlds. Both the empowerment agent and its curious competitor seem to benefit to similar extents from their intrinsic rewards. This provides some experimental support to the conjecture that empowerment can be used to drive exploration.

* 6 pages, 3 figures, to be published in proceedings of the International Conference on Development and Learning 2021  

Share this with someone who'll enjoy it: