Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox


Single-Agent Policy Tree Search With Guarantees

Nov 28, 2018
Laurent Orseau, Levi H. S. Lelis, Tor Lattimore, Théophane Weber



We introduce two novel tree search algorithms that use a policy to guide search. The first algorithm is a best-first enumeration that uses a cost function that allows us to prove an upper bound on the number of nodes to be expanded before reaching a goal state. We show that this best-first algorithm is particularly well suited for `needle-in-a-haystack' problems. The second algorithm is based on sampling and we prove an upper bound on the expected number of nodes it expands before reaching a set of goal states. We show that this algorithm is better suited for problems where many paths lead to a goal. We validate these tree search algorithms on 1,000 computer-generated levels of Sokoban, where the policy used to guide the search comes from a neural network trained using A3C. Our results show that the policy tree search algorithms we introduce are competitive with a state-of-the-art domain-independent planner that uses heuristic search.

* 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montr\'eal, Canada 


Share this with someone who'll enjoy it:

   Access Paper Source



Share this with someone who'll enjoy it: