Alert button
Picture for Min Cheng

Min Cheng

Alert button

Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation

Jul 17, 2023
Ruida Zhou, Tao Liu, Min Cheng, Dileep Kalathil, P. R. Kumar, Chao Tian

Figure 1 for Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation
Figure 2 for Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation
Figure 3 for Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation
Figure 4 for Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation

We study robust reinforcement learning (RL) with the goal of determining a well-performing policy that is robust against model mismatch between the training simulator and the testing environment. Previous policy-based robust RL algorithms mainly focus on the tabular setting under uncertainty sets that facilitate robust policy evaluation, but are no longer tractable when the number of states scales up. To this end, we propose two novel uncertainty set formulations, one based on double sampling and the other on an integral probability metric. Both make large-scale robust RL tractable even when one only has access to a simulator. We propose a robust natural actor-critic (RNAC) approach that incorporates the new uncertainty sets and employs function approximation. We provide finite-time convergence guarantees for the proposed RNAC algorithm to the optimal robust policy within the function approximation error. Finally, we demonstrate the robust performance of the policy learned by our proposed RNAC approach in multiple MuJoCo environments and a real-world TurtleBot navigation task.

Viaarxiv icon

MMED: A Multi-domain and Multi-modality Event Dataset

Apr 09, 2019
Zhenguo Yang, Zehang Lin, Min Cheng, Qing Li, Wenyin Liu

Figure 1 for MMED: A Multi-domain and Multi-modality Event Dataset
Figure 2 for MMED: A Multi-domain and Multi-modality Event Dataset
Figure 3 for MMED: A Multi-domain and Multi-modality Event Dataset
Figure 4 for MMED: A Multi-domain and Multi-modality Event Dataset

In this work, we construct and release a multi-domain and multi-modality event dataset (MMED), containing 25,165 textual news articles collected from hundreds of news media sites (e.g., Yahoo News, Google News, CNN News.) and 76,516 image posts shared on Flickr social media, which are annotated according to 412 real-world events. The dataset is collected to explore the problem of organizing heterogeneous data contributed by professionals and amateurs in different data domains, and the problem of transferring event knowledge obtained from one data domain to heterogeneous data domain, thus summarizing the data with different contributors. We hope that the release of the MMED dataset can stimulate innovate research on related challenging problems, such as event discovery, cross-modal (event) retrieval, and visual question answering, etc.

Viaarxiv icon