Get our free extension to see links to code for papers anywhere online!

 Add to Chrome

 Add to Firefox

CatalyzeX Code Finder - Browser extension linking code for ML papers across the web! | Product Hunt Embed

A Unified Switching System Perspective and O.D.E. Analysis of Q-Learning Algorithms

Dec 04, 2019
Donghwan Lee, Niao He



In this paper, we introduce a unified framework for analyzing a large family of Q-learning algorithms, based on switching system perspectives and ODE-based stochastic approximation. We show that the nonlinear ODE models associated with these Q-learning algorithms can be formulated as switched linear systems, and analyze their asymptotic stability by leveraging existing switching system theories. Our approach provides the first O.D.E. analysis of the asymptotic convergences of various Q-learning algorithms, including asynchronous Q-learning, averaging Q-learning, double Q-learning with or without regularization. We also extend the approach to analyze Q-learning with linear function approximation and derive a new sufficient condition for its convergence.



Share this with someone who'll enjoy it:

   Access Paper Source



Share this with someone who'll enjoy it: