Neural networks with motivation

Motivational salience is a mechanism that determines an organism's current level of attraction to or repulsion from a particular object, event, or outcome. Motivational salience is described by modulating the reward by an externally controlled parameter that remains constant within a single behavioral episode. The vector of perceived values of various outcomes determines motivation of an organism toward different goals. Organism's behavior should be able to adapt to the varying-in-time motivation vector. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal behavior for different dynamically changing motivation vectors. First, we show that Q-learning neural networks can learn to navigate towards variable goals whose relative salience is determined by a multidimensional motivational vector. Second, we show that a Q-learning network with motivation can learn complex behaviors towards several goals distributed in an environment. Finally, we show that firing patterns displayed by neurons in the ventral pallidum, a basal ganglia structure playing a crucial role in motivated behaviors, are similar to the responses of neurons in recurrent neural networks trained in similar conditions. Similarly to the pallidum neurons, artificial neural nets contain two different classes of neurons, tuned to reward and punishment. We conclude that reinforcement learning networks can efficiently learn optimal behavior in conditions when reward values are modulated by external motivational processes with arbitrary dynamics. Motivational salience can be viewed as a general-purpose model-free method identifying and capturing changes in subjective or objective values of multiple rewards. Networks with motivation may also be parts of a larger hierarchical reinforcement learning system in the brain.