Alert button

On the Robustness, Generalization, and Forgetting of Shape-Texture Debiased Continual Learning

Nov 21, 2022
Zenglin Shi, Ying Sun, Joo Hwee Lim, Mengmi Zhang

Figure 1 for On the Robustness, Generalization, and Forgetting of Shape-Texture Debiased Continual Learning
Figure 2 for On the Robustness, Generalization, and Forgetting of Shape-Texture Debiased Continual Learning
Figure 3 for On the Robustness, Generalization, and Forgetting of Shape-Texture Debiased Continual Learning
Figure 4 for On the Robustness, Generalization, and Forgetting of Shape-Texture Debiased Continual Learning

Share this with someone who'll enjoy it:

Tremendous progress has been made in continual learning to maintain good performance on old tasks when learning new tasks by tackling the catastrophic forgetting problem of neural networks. This paper advances continual learning by further considering its out-of-distribution robustness, in response to the vulnerability of continually trained models to distribution shifts (e.g., due to data corruptions and domain shifts) in inference. To this end, we propose shape-texture debiased continual learning. The key idea is to learn generalizable and robust representations for each task with shape-texture debiased training. In order to transform standard continual learning to shape-texture debiased continual learning, we propose shape-texture debiased data generation and online shape-texture debiased self-distillation. Experiments on six datasets demonstrate the benefits of our approach in improving generalization and robustness, as well as reducing forgetting. Our analysis on the flatness of the loss landscape explains the advantages. Moreover, our approach can be easily combined with new advanced architectures such as vision transformer, and applied to more challenging scenarios such as exemplar-free continual learning.

View paper onarxiv icon

Share this with someone who'll enjoy it: