Detecting Universal Trigger's Adversarial Attack with Honeypot

The Universal Trigger (UniTrigger) is a recently-proposed powerful adversarial textual attack method. Utilizing a learning-based mechanism, UniTrigger can generate a fixed phrase that when added to any benign inputs, can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. To defend against this new attack method that may cause significant harm, we borrow the "honeypot" concept from the cybersecurity community and propose DARCY, a honeypot-based defense framework. DARCY adaptively searches and injects multiple trapdoors into an NN model to "bait and catch" potential attacks. Through comprehensive experiments across five public datasets, we demonstrate that DARCY detects UniTrigger's adversarial attacks with up to 99% TPR and less than 1% FPR in most cases, while showing a difference of only around 2% of F1 score on average in predicting for clean inputs. We also show that DARCY with multiple trapdoors is robust under different assumptions with respect to attackers' knowledge and skills.