Alert button

Scaling in Depth: Unlocking Robustness Certification on ImageNet

Jan 29, 2023
Kai Hu, Andy Zou, Zifan Wang, Klas Leino, Matt Fredrikson

Figure 1 for Scaling in Depth: Unlocking Robustness Certification on ImageNet
Figure 2 for Scaling in Depth: Unlocking Robustness Certification on ImageNet
Figure 3 for Scaling in Depth: Unlocking Robustness Certification on ImageNet
Figure 4 for Scaling in Depth: Unlocking Robustness Certification on ImageNet

Share this with someone who'll enjoy it:

Notwithstanding the promise of Lipschitz-based approaches to \emph{deterministically} train and certify robust deep networks, the state-of-the-art results only make successful use of feed-forward Convolutional Networks (ConvNets) on low-dimensional data, e.g. CIFAR-10. Because ConvNets often suffer from vanishing gradients when going deep, large-scale datasets with many classes, e.g., ImageNet, have remained out of practical reach. This paper investigates ways to scale up certifiably robust training to Residual Networks (ResNets). First, we introduce the \emph{Linear ResNet} (LiResNet) architecture, which utilizes a new residual block designed to facilitate \emph{tighter} Lipschitz bounds compared to a conventional residual block. Second, we introduce Efficient Margin MAximization (EMMA), a loss function that stabilizes robust training by simultaneously penalizing worst-case adversarial examples from \emph{all} classes. Combining LiResNet and EMMA, we achieve new \emph{state-of-the-art} robust accuracy on CIFAR-10/100 and Tiny-ImageNet under $\ell_2$-norm-bounded perturbations. Moreover, for the first time, we are able to scale up deterministic robustness guarantees to ImageNet, bringing hope to the possibility of applying deterministic certification to real-world applications.

View paper onarxiv icon

Share this with someone who'll enjoy it: