Alert button

Fast Global Convergence of Policy Optimization for Constrained MDPs

Oct 31, 2021
Tao Liu, Ruida Zhou, Dileep Kalathil, P. R. Kumar, Chao Tian

Figure 1 for Fast Global Convergence of Policy Optimization for Constrained MDPs

Share this with someone who'll enjoy it:

We address the issue of safety in reinforcement learning. We pose the problem in a discounted infinite-horizon constrained Markov decision process framework. Existing results have shown that gradient-based methods are able to achieve an $\mathcal{O}(1/\sqrt{T})$ global convergence rate both for the optimality gap and the constraint violation. We exhibit a natural policy gradient-based algorithm that has a faster convergence rate $\mathcal{O}(\log(T)/T)$ for both the optimality gap and the constraint violation. When Slater's condition is satisfied and known a priori, zero constraint violation can be further guaranteed for a sufficiently large $T$ while maintaining the same convergence rate.

View paper onarxiv icon

Share this with someone who'll enjoy it: