Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Richard S. Sutton

Richard S. Sutton

Does Standard Backpropagation Forget Less Catastrophically Than Adam?


Feb 20, 2021
Dylan R. Ashley, Sina Ghiassian, Richard S. Sutton

Figure 1 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
Figure 2 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
Figure 3 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
Figure 4 for Does Standard Backpropagation Forget Less Catastrophically Than Adam?
* 8 pages in main text + 3 pages of references + 6 pages of appendices, 6 figures in main text + 14 figures in appendices, 1 table in main text + 3 tables in appendices; source code available at https://github.com/dylanashley/catastrophic-forgetting/tree/arxiv 
  Access Paper or Ask Questions

Average-Reward Off-Policy Policy Evaluation with Function Approximation


Jan 08, 2021
Shangtong Zhang, Yi Wan, Richard S. Sutton, Shimon Whiteson

Figure 1 for Average-Reward Off-Policy Policy Evaluation with Function Approximation
Figure 2 for Average-Reward Off-Policy Policy Evaluation with Function Approximation
Figure 3 for Average-Reward Off-Policy Policy Evaluation with Function Approximation
Figure 4 for Average-Reward Off-Policy Policy Evaluation with Function Approximation

  Access Paper or Ask Questions

Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning


Oct 28, 2020
Kenny Young, Richard S. Sutton

Figure 1 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning
Figure 2 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning
Figure 3 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning
Figure 4 for Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning

  Access Paper or Ask Questions

Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI


Aug 27, 2020
Katya Kudashkina, Patrick M. Pilarski, Richard S. Sutton

Figure 1 for Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI
* Currently under review 
  Access Paper or Ask Questions

Inverse Policy Evaluation for Value-based Sequential Decision-making


Aug 26, 2020
Alan Chan, Kris de Asis, Richard S. Sutton

Figure 1 for Inverse Policy Evaluation for Value-based Sequential Decision-making
Figure 2 for Inverse Policy Evaluation for Value-based Sequential Decision-making
Figure 3 for Inverse Policy Evaluation for Value-based Sequential Decision-making
Figure 4 for Inverse Policy Evaluation for Value-based Sequential Decision-making
* Submitted to NeurIPS 2020 
  Access Paper or Ask Questions

Learning and Planning in Average-Reward Markov Decision Processes


Jun 29, 2020
Yi Wan, Abhishek Naik, Richard S. Sutton

Figure 1 for Learning and Planning in Average-Reward Markov Decision Processes
Figure 2 for Learning and Planning in Average-Reward Markov Decision Processes
Figure 3 for Learning and Planning in Average-Reward Markov Decision Processes
Figure 4 for Learning and Planning in Average-Reward Markov Decision Processes

  Access Paper or Ask Questions

Learning Sparse Representations Incrementally in Deep Reinforcement Learning


Dec 09, 2019
J. Fernando Hernandez-Garcia, Richard S. Sutton

Figure 1 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning
Figure 2 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning
Figure 3 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning
Figure 4 for Learning Sparse Representations Incrementally in Deep Reinforcement Learning

  Access Paper or Ask Questions

Discounted Reinforcement Learning is Not an Optimization Problem


Nov 16, 2019
Abhishek Naik, Roshan Shariff, Niko Yasui, Richard S. Sutton

* Accepted for presentation at the Optimization Foundations of Reinforcement Learning Workshop at NeurIPS 2019 
  Access Paper or Ask Questions

Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning


Sep 09, 2019
Kristopher De Asis, Alan Chan, Silviu Pitis, Richard S. Sutton, Daniel Graves

Figure 1 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning
Figure 2 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning
Figure 3 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning
Figure 4 for Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning

  Access Paper or Ask Questions

Planning with Expectation Models


Apr 03, 2019
Yi Wan, Muhammad Zaheer, Adam White, Martha White, Richard S. Sutton

Figure 1 for Planning with Expectation Models
Figure 2 for Planning with Expectation Models
Figure 3 for Planning with Expectation Models

  Access Paper or Ask Questions

Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning


Mar 08, 2019
Alex Kearney, Vivek Veeriah, Jaden Travnik, Patrick M. Pilarski, Richard S. Sutton

Figure 1 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning
Figure 2 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning
Figure 3 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning
Figure 4 for Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning

  Access Paper or Ask Questions

Should All Temporal Difference Learning Use Emphasis?


Mar 01, 2019
Xiang Gu, Sina Ghiassian, Richard S. Sutton

Figure 1 for Should All Temporal Difference Learning Use Emphasis?
Figure 2 for Should All Temporal Difference Learning Use Emphasis?
Figure 3 for Should All Temporal Difference Learning Use Emphasis?
Figure 4 for Should All Temporal Difference Learning Use Emphasis?

  Access Paper or Ask Questions

Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target


Feb 07, 2019
J. Fernando Hernandez-Garcia, Richard S. Sutton

Figure 1 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target
Figure 2 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target
Figure 3 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target
Figure 4 for Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target

  Access Paper or Ask Questions

Online Off-policy Prediction


Nov 06, 2018
Sina Ghiassian, Andrew Patterson, Martha White, Richard S. Sutton, Adam White

Figure 1 for Online Off-policy Prediction
Figure 2 for Online Off-policy Prediction
Figure 3 for Online Off-policy Prediction
Figure 4 for Online Off-policy Prediction
* 68 pages 
  Access Paper or Ask Questions

Predicting Periodicity with Temporal Difference Learning


Sep 20, 2018
Kristopher De Asis, Brendan Bennett, Richard S. Sutton

Figure 1 for Predicting Periodicity with Temporal Difference Learning
Figure 2 for Predicting Periodicity with Temporal Difference Learning
Figure 3 for Predicting Periodicity with Temporal Difference Learning
Figure 4 for Predicting Periodicity with Temporal Difference Learning

  Access Paper or Ask Questions

Two geometric input transformation methods for fast online reinforcement learning with neural nets


Sep 06, 2018
Sina Ghiassian, Huizhen Yu, Banafsheh Rafiee, Richard S. Sutton

Figure 1 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
Figure 2 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
Figure 3 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
Figure 4 for Two geometric input transformation methods for fast online reinforcement learning with neural nets
* 16 pages 
  Access Paper or Ask Questions

Per-decision Multi-step Temporal Difference Learning with Control Variates


Jul 05, 2018
Kristopher De Asis, Richard S. Sutton

Figure 1 for Per-decision Multi-step Temporal Difference Learning with Control Variates
Figure 2 for Per-decision Multi-step Temporal Difference Learning with Control Variates
Figure 3 for Per-decision Multi-step Temporal Difference Learning with Control Variates
Figure 4 for Per-decision Multi-step Temporal Difference Learning with Control Variates
* (2018). In Conference on Uncertainty in Artificial Intelligence. http://auai.org/uai2018/proceedings/papers/282.pdf 
  Access Paper or Ask Questions

Multi-step Reinforcement Learning: A Unifying Algorithm


Jun 11, 2018
Kristopher De Asis, J. Fernando Hernandez-Garcia, G. Zacharias Holland, Richard S. Sutton

Figure 1 for Multi-step Reinforcement Learning: A Unifying Algorithm
Figure 2 for Multi-step Reinforcement Learning: A Unifying Algorithm
Figure 3 for Multi-step Reinforcement Learning: A Unifying Algorithm
Figure 4 for Multi-step Reinforcement Learning: A Unifying Algorithm
* (2018). In AAAI Conference on Artificial Intelligence. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16294 
 * Appeared at the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) 
  Access Paper or Ask Questions

Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling


Jun 01, 2018
Kenny J. Young, Richard S. Sutton, Shuo Yang

Figure 1 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
Figure 2 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
Figure 3 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
Figure 4 for Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling

  Access Paper or Ask Questions

A Deeper Look at Experience Replay


Apr 30, 2018
Shangtong Zhang, Richard S. Sutton

Figure 1 for A Deeper Look at Experience Replay
Figure 2 for A Deeper Look at Experience Replay
Figure 3 for A Deeper Look at Experience Replay
Figure 4 for A Deeper Look at Experience Replay
* NIPS 2017 Deep Reinforcement Learning Symposium 
  Access Paper or Ask Questions

TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent


Apr 10, 2018
Alex Kearney, Vivek Veeriah, Jaden B. Travnik, Richard S. Sutton, Patrick M. Pilarski

Figure 1 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
Figure 2 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
Figure 3 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
Figure 4 for TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descent
* Version as submitted to the 31st Conference on Neural Information Processing Systems (NIPS 2017) on May 19, 2017. 9 pages, 5 figures. Extended version in preparation for journal submission 
  Access Paper or Ask Questions

Reactive Reinforcement Learning in Asynchronous Environments


Feb 16, 2018
Jaden B. Travnik, Kory W. Mathewson, Richard S. Sutton, Patrick M. Pilarski

Figure 1 for Reactive Reinforcement Learning in Asynchronous Environments
Figure 2 for Reactive Reinforcement Learning in Asynchronous Environments
Figure 3 for Reactive Reinforcement Learning in Asynchronous Environments
Figure 4 for Reactive Reinforcement Learning in Asynchronous Environments
* 11 pages, 7 figures, currently under journal peer review 
  Access Paper or Ask Questions

Directly Estimating the Variance of the 位-Return Using Temporal-Difference Methods


Feb 14, 2018
Craig Sherstan, Brendan Bennett, Kenny Young, Dylan R. Ashley, Adam White, Martha White, Richard S. Sutton

Figure 1 for Directly Estimating the Variance of the 位-Return Using Temporal-Difference Methods
Figure 2 for Directly Estimating the Variance of the 位-Return Using Temporal-Difference Methods
Figure 3 for Directly Estimating the Variance of the 位-Return Using Temporal-Difference Methods
Figure 4 for Directly Estimating the Variance of the 位-Return Using Temporal-Difference Methods

  Access Paper or Ask Questions

Communicative Capital for Prosthetic Agents


Nov 10, 2017
Patrick M. Pilarski, Richard S. Sutton, Kory W. Mathewson, Craig Sherstan, Adam S. R. Parker, Ann L. Edwards

Figure 1 for Communicative Capital for Prosthetic Agents
Figure 2 for Communicative Capital for Prosthetic Agents
Figure 3 for Communicative Capital for Prosthetic Agents
Figure 4 for Communicative Capital for Prosthetic Agents
* 33 pages, 10 figures; unpublished technical report undergoing peer review 
  Access Paper or Ask Questions

A First Empirical Study of Emphatic Temporal Difference Learning


May 12, 2017
Sina Ghiassian, Banafsheh Rafiee, Richard S. Sutton

Figure 1 for A First Empirical Study of Emphatic Temporal Difference Learning
Figure 2 for A First Empirical Study of Emphatic Temporal Difference Learning
* 5 pages, Accepted to NIPS Continual Learning and Deep Networks workshop, 2016 
  Access Paper or Ask Questions

GQ($位$) Quick Reference and Implementation Guide


May 10, 2017
Adam White, Richard S. Sutton


  Access Paper or Ask Questions

Integral Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space


May 09, 2017
Jae Young Lee, Richard S. Sutton

Figure 1 for Integral Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space
Figure 2 for Integral Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space
Figure 3 for Integral Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space
Figure 4 for Integral Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space
* 20 pages, 2 figures (i.e., 6 sub-figures), 2 tables, 5 main ideal algorithms, and 1 algorithm for implementation. For a summary, a short simplified RLDM-conf. version is available at  
  Access Paper or Ask Questions