Alert button

Neuro-Symbolic World Models for Adapting to Open World Novelty

Jan 16, 2023
Jonathan Balloch, Zhiyu Lin, Robert Wright, Xiangyu Peng, Mustafa Hussain, Aarun Srinivas, Julia Kim, Mark O. Riedl

Figure 1 for Neuro-Symbolic World Models for Adapting to Open World Novelty
Figure 2 for Neuro-Symbolic World Models for Adapting to Open World Novelty
Figure 3 for Neuro-Symbolic World Models for Adapting to Open World Novelty
Figure 4 for Neuro-Symbolic World Models for Adapting to Open World Novelty

Share this with someone who'll enjoy it:

Open-world novelty--a sudden change in the mechanics or properties of an environment--is a common occurrence in the real world. Novelty adaptation is an agent's ability to improve its policy performance post-novelty. Most reinforcement learning (RL) methods assume that the world is a closed, fixed process. Consequentially, RL policies adapt inefficiently to novelties. To address this, we introduce WorldCloner, an end-to-end trainable neuro-symbolic world model for rapid novelty adaptation. WorldCloner learns an efficient symbolic representation of the pre-novelty environment transitions, and uses this transition model to detect novelty and efficiently adapt to novelty in a single-shot fashion. Additionally, WorldCloner augments the policy learning process using imagination-based adaptation, where the world model simulates transitions of the post-novelty environment to help the policy adapt. By blending ''imagined'' transitions with interactions in the post-novelty environment, performance can be recovered with fewer total environment interactions. Using environments designed for studying novelty in sequential decision-making problems, we show that the symbolic world model helps its neural policy adapt more efficiently than model-based and model-based neural-only reinforcement learning methods.

* 9 pages, 8 figures, Extended Abstract accepted for presentation at AAMAS 2023  
View paper onarxiv icon

Share this with someone who'll enjoy it: