Adversarial Sample Detection for Deep Neural Network through Model Mutation TestingTechnical Track
Deep neural networks (DNN) have been shown to be useful in a wide range of applications. However, they are also known to be vulnerable to adversarial samples. By transforming a normal sample with some carefully crafted human non-perceptible perturbations, even highly accurate DNN makes wrong decisions. Multiple defense mechanisms have been proposed which aim to hinder the generation of such adversarial samples. However, a recent work show that most of them are ineffective. In this work, we propose an alternative approach to detect adversarial samples at runtime. Our main observation is that adversarial samples are much more sensitive than normal samples if we impose random mutations on the DNN. We thus first propose a measure of `sensitivity’ and show empirically that normal samples and adversarial samples have distinguishable sensitivity. We then integrate statistical model checking and mutation testing to check whether an input sample is normal or adversarial at runtime by measuring its sensitivity. We evaluated our approach on the MNIST and CIFAR10 dataset. The results show that our approach detects adversarial samples generated by state-of-art attacking methods efficiently and accurately.
Fri 31 May
16:00 - 17:20: Papers - Testing and Analysis: Domain-Specific Approaches at Place du Canada Chair(s): Gregory GayUniversity of South Carolina, Chalmers | University of Gothenburg | ||||||||||||||||||||||||||||||||||||||||||
16:00 - 16:20 Talk | Pre-print Media Attached | |||||||||||||||||||||||||||||||||||||||||
16:20 - 16:40 Talk | Jingyi WangNational University of Singapore, Singapore, Guoliang DongComputer College of Zhejiang University, Jun SunSingapore Management University, Singapore, Xinyu WangZhejiang University, Peixin ZhangZhejiang University | |||||||||||||||||||||||||||||||||||||||||
16:40 - 16:50 Talk | Chunhui WangUniversity of Luxembourg, Fabrizio PastoreUniversity of Luxembourg, Lionel BriandSnT Centre/University of Luxembourg | |||||||||||||||||||||||||||||||||||||||||
16:50 - 17:10 Talk | Yuting ChenShanghai Jiao Tong University, Ting SuNanyang Technological University, Singapore, Zhendong SuETH Zurich | |||||||||||||||||||||||||||||||||||||||||
17:10 - 17:20 Talk |