Alert button

Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation

Sep 26, 2013
Marek Petrik, Dharmashankar Subramanian, Janusz Marecki

Figure 1 for Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation
Figure 2 for Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation
Figure 3 for Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation
Figure 4 for Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation

Share this with someone who'll enjoy it:

We propose solution methods for previously-unsolved constrained MDPs in which actions can continuously modify the transition probabilities within some acceptable sets. While many methods have been proposed to solve regular MDPs with large state sets, there are few practical approaches for solving constrained MDPs with large action sets. In particular, we show that the continuous action sets can be replaced by their extreme points when the rewards are linear in the modulation. We also develop a tractable optimization formulation for concave reward functions and, surprisingly, also extend it to non- concave reward functions by using their concave envelopes. We evaluate the effectiveness of the approach on the problem of managing delinquencies in a portfolio of loans.

* Appears in Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI2013)  
View paper onarxiv icon

Share this with someone who'll enjoy it: