Alert button

RL-DARTS: Differentiable Architecture Search for Reinforcement Learning

Jun 04, 2021
Yingjie Miao, Xingyou Song, Daiyi Peng, Summer Yue, Eugene Brevdo, Aleksandra Faust

Figure 1 for RL-DARTS: Differentiable Architecture Search for Reinforcement Learning
Figure 2 for RL-DARTS: Differentiable Architecture Search for Reinforcement Learning
Figure 3 for RL-DARTS: Differentiable Architecture Search for Reinforcement Learning
Figure 4 for RL-DARTS: Differentiable Architecture Search for Reinforcement Learning

Share this with someone who'll enjoy it:

We introduce RL-DARTS, one of the first applications of Differentiable Architecture Search (DARTS) in reinforcement learning (RL) to search for convolutional cells, applied to the Procgen benchmark. We outline the initial difficulties of applying neural architecture search techniques in RL, and demonstrate that by simply replacing the image encoder with a DARTS supernet, our search method is sample-efficient, requires minimal extra compute resources, and is also compatible with off-policy and on-policy RL algorithms, needing only minor changes in preexisting code. Surprisingly, we find that the supernet can be used as an actor for inference to generate replay data in standard RL training loops, and thus train end-to-end. Throughout this training process, we show that the supernet gradually learns better cells, leading to alternative architectures which can be highly competitive against manually designed policies, but also verify previous design choices for RL policies.

* 19 pages total, 17 figures  
View paper onarxiv iconopen_review iconOpenReview

Share this with someone who'll enjoy it: