Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Richard S. Sutton

Richard S. Sutton

Average-Reward Learning and Planning with Options


Oct 26, 2021
Yi Wan, Abhishek Naik, Richard S. Sutton

Figure 1 for Average-Reward Learning and Planning with Options
Figure 2 for Average-Reward Learning and Planning with Options
Figure 3 for Average-Reward Learning and Planning with Options
Figure 4 for Average-Reward Learning and Planning with Options

  Access Paper or Ask Questions

An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment


Sep 10, 2021
Sina Ghiassian, Richard S. Sutton

Figure 1 for An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment
Figure 2 for An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment
Figure 3 for An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment
Figure 4 for An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment
* 13 pages 
  Access Paper or Ask Questions

Continual Backprop: Stochastic Gradient Descent with Persistent Randomness


Aug 13, 2021
Shibhansh Dohare, A. Rupam Mahmood, Richard S. Sutton

Figure 1 for Continual Backprop: Stochastic Gradient Descent with Persistent Randomness
Figure 2 for Continual Backprop: Stochastic Gradient Descent with Persistent Randomness
Figure 3 for Continual Backprop: Stochastic Gradient Descent with Persistent Randomness
Figure 4 for Continual Backprop: Stochastic Gradient Descent with Persistent Randomness

  Access Paper or Ask Questions

An Empirical Comparison of Off-policy Prediction Learning Algorithms on the Collision Task


Jun 11, 2021
Sina Ghiassian, Richard S. Sutton

Figure 1 for An Empirical Comparison of Off-policy Prediction Learning Algorithms on the Collision Task
Figure 2 for An Empirical Comparison of Off-policy Prediction Learning Algorithms on the Collision Task
Figure 3 for An Empirical Comparison of Off-policy Prediction Learning Algorithms on the Collision Task
Figure 4 for An Empirical Comparison of Off-policy Prediction Learning Algorithms on the Collision Task

  Access Paper or Ask Questions

Planning with Expectation Models for Control


Apr 17, 2021
Katya Kudashkina, Yi Wan, Abhishek Naik, Richard S. Sutton

Figure 1 for Planning with Expectation Models for Control
Figure 2 for Planning with Expectation Models for Control
Figure 3 for Planning with Expectation Models for Control
Figure 4 for Planning with Expectation Models for Control

  Access Paper or Ask Questions

Does Standard Backpropagation Forget Less Catastrophically Than Adam?


Feb 20, 2021
Dylan R. Ashley, Sina Ghiassian, Richard S. Sutton

Figure 1 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
Figure 2 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
Figure 3 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
Figure 4 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
* 8 pages in main text + 3 pages of references + 6 pages of appendices, 6 figures in main text + 14 figures in appendices, 1 table in main text + 3 tables in appendices; source code available at https://github.com/dylanashley/catastrophic-forgetting/tree/arxiv 
  Access Paper or Ask Questions

Average-Reward Off-Policy Policy Evaluation with Function Approximation


Jan 08, 2021
Shangtong Zhang, Yi Wan, Richard S. Sutton, Shimon Whiteson

Figure 1 for Average-Reward Off-Policy Policy Evaluation with Function Approximation
Figure 2 for Average-Reward Off-Policy Policy Evaluation with Function Approximation
Figure 3 for Average-Reward Off-Policy Policy Evaluation with Function Approximation
Figure 4 for Average-Reward Off-Policy Policy Evaluation with Function Approximation

  Access Paper or Ask Questions

Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning


Oct 28, 2020
Kenny Young, Richard S. Sutton

Figure 1 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning
Figure 2 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning
Figure 3 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning
Figure 4 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning

  Access Paper or Ask Questions

Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI


Aug 27, 2020
Katya Kudashkina, Patrick M. Pilarski, Richard S. Sutton

Figure 1 for Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI
* Currently under review 
  Access Paper or Ask Questions

Inverse Policy Evaluation for Value-based Sequential Decision-making


Aug 26, 2020
Alan Chan, Kris de Asis, Richard S. Sutton

Figure 1 for Inverse Policy Evaluation for Value-based Sequential Decision-making
Figure 2 for Inverse Policy Evaluation for Value-based Sequential Decision-making
Figure 3 for Inverse Policy Evaluation for Value-based Sequential Decision-making
Figure 4 for Inverse Policy Evaluation for Value-based Sequential Decision-making
* Submitted to NeurIPS 2020 
  Access Paper or Ask Questions

Learning and Planning in Average-Reward Markov Decision Processes


Jun 29, 2020
Yi Wan, Abhishek Naik, Richard S. Sutton

Figure 1 for Learning and Planning in Average-Reward Markov Decision Processes
Figure 2 for Learning and Planning in Average-Reward Markov Decision Processes
Figure 3 for Learning and Planning in Average-Reward Markov Decision Processes
Figure 4 for Learning and Planning in Average-Reward Markov Decision Processes

  Access Paper or Ask Questions

Learning Sparse Representations Incrementally in Deep Reinforcement Learning


Dec 09, 2019
J. Fernando Hernandez-Garcia, Richard S. Sutton

Figure 1 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning
Figure 2 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning
Figure 3 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning
Figure 4 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning

  Access Paper or Ask Questions

Discounted Reinforcement Learning is Not an Optimization Problem


Nov 16, 2019
Abhishek Naik, Roshan Shariff, Niko Yasui, Richard S. Sutton

* Accepted for presentation at the Optimization Foundations of Reinforcement Learning Workshop at NeurIPS 2019 
  Access Paper or Ask Questions

Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning


Sep 09, 2019
Kristopher De Asis, Alan Chan, Silviu Pitis, Richard S. Sutton, Daniel Graves

Figure 1 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning
Figure 2 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning
Figure 3 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning
Figure 4 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning

  Access Paper or Ask Questions

Planning with Expectation Models


Apr 03, 2019
Yi Wan, Muhammad Zaheer, Adam White, Martha White, Richard S. Sutton

Figure 1 for Planning with Expectation Models
Figure 2 for Planning with Expectation Models
Figure 3 for Planning with Expectation Models

  Access Paper or Ask Questions

Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning


Mar 08, 2019
Alex Kearney, Vivek Veeriah, Jaden Travnik, Patrick M. Pilarski, Richard S. Sutton

Figure 1 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning
Figure 2 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning
Figure 3 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning
Figure 4 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning

  Access Paper or Ask Questions

Should All Temporal Difference Learning Use Emphasis?


Mar 01, 2019
Xiang Gu, Sina Ghiassian, Richard S. Sutton

Figure 1 for Should All Temporal Difference Learning Use Emphasis?
Figure 2 for Should All Temporal Difference Learning Use Emphasis?
Figure 3 for Should All Temporal Difference Learning Use Emphasis?
Figure 4 for Should All Temporal Difference Learning Use Emphasis?

  Access Paper or Ask Questions

Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target


Feb 07, 2019
J. Fernando Hernandez-Garcia, Richard S. Sutton

Figure 1 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target
Figure 2 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target
Figure 3 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target
Figure 4 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target

  Access Paper or Ask Questions

Online Off-policy Prediction


Nov 06, 2018
Sina Ghiassian, Andrew Patterson, Martha White, Richard S. Sutton, Adam White

Figure 1 for Online Off-policy Prediction
Figure 2 for Online Off-policy Prediction
Figure 3 for Online Off-policy Prediction
Figure 4 for Online Off-policy Prediction
* 68 pages 
  Access Paper or Ask Questions

Predicting Periodicity with Temporal Difference Learning


Sep 20, 2018
Kristopher De Asis, Brendan Bennett, Richard S. Sutton

Figure 1 for Predicting Periodicity with Temporal Difference Learning
Figure 2 for Predicting Periodicity with Temporal Difference Learning
Figure 3 for Predicting Periodicity with Temporal Difference Learning
Figure 4 for Predicting Periodicity with Temporal Difference Learning

  Access Paper or Ask Questions

Two geometric input transformation methods for fast online reinforcement learning with neural nets


Sep 06, 2018
Sina Ghiassian, Huizhen Yu, Banafsheh Rafiee, Richard S. Sutton

Figure 1 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
Figure 2 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
Figure 3 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
Figure 4 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
* 16 pages 
  Access Paper or Ask Questions

Per-decision Multi-step Temporal Difference Learning with Control Variates


Jul 05, 2018
Kristopher De Asis, Richard S. Sutton

Figure 1 for Per-decision Multi-step Temporal Difference Learning with Control Variates
Figure 2 for Per-decision Multi-step Temporal Difference Learning with Control Variates
Figure 3 for Per-decision Multi-step Temporal Difference Learning with Control Variates
Figure 4 for Per-decision Multi-step Temporal Difference Learning with Control Variates
* (2018). In Conference on Uncertainty in Artificial Intelligence. http://auai.org/uai2018/proceedings/papers/282.pdf 
  Access Paper or Ask Questions

Multi-step Reinforcement Learning: A Unifying Algorithm


Jun 11, 2018
Kristopher De Asis, J. Fernando Hernandez-Garcia, G. Zacharias Holland, Richard S. Sutton

Figure 1 for Multi-step Reinforcement Learning: A Unifying Algorithm
Figure 2 for Multi-step Reinforcement Learning: A Unifying Algorithm
Figure 3 for Multi-step Reinforcement Learning: A Unifying Algorithm
Figure 4 for Multi-step Reinforcement Learning: A Unifying Algorithm
* (2018). In AAAI Conference on Artificial Intelligence. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16294 
 * Appeared at the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) 
  Access Paper or Ask Questions

Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling


Jun 01, 2018
Kenny J. Young, Richard S. Sutton, Shuo Yang

Figure 1 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
Figure 2 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
Figure 3 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
Figure 4 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling

  Access Paper or Ask Questions

A Deeper Look at Experience Replay


Apr 30, 2018
Shangtong Zhang, Richard S. Sutton

Figure 1 for A Deeper Look at Experience Replay
Figure 2 for A Deeper Look at Experience Replay
Figure 3 for A Deeper Look at Experience Replay
Figure 4 for A Deeper Look at Experience Replay
* NIPS 2017 Deep Reinforcement Learning Symposium 
  Access Paper or Ask Questions

TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent


Apr 10, 2018
Alex Kearney, Vivek Veeriah, Jaden B. Travnik, Richard S. Sutton, Patrick M. Pilarski

Figure 1 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
Figure 2 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
Figure 3 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
Figure 4 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
* Version as submitted to the 31st Conference on Neural Information Processing Systems (NIPS 2017) on May 19, 2017. 9 pages, 5 figures. Extended version in preparation for journal submission 
  Access Paper or Ask Questions