Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense
Escalating advancements in artificial intelligence (AI) has prompted significant security concerns, especially with its increasing commercialization. This necessitates research on safety measures to securely utilize AI models. Existing AI models are vulnerable to adversarial attacks, which are a spe...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/23/10872 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846124490258382848 |
|---|---|
| author | Inpyo Hong Sokjoon Lee |
| author_facet | Inpyo Hong Sokjoon Lee |
| author_sort | Inpyo Hong |
| collection | DOAJ |
| description | Escalating advancements in artificial intelligence (AI) has prompted significant security concerns, especially with its increasing commercialization. This necessitates research on safety measures to securely utilize AI models. Existing AI models are vulnerable to adversarial attacks, which are a specific form of assault methodology. Although various countermeasures have been explored, practical defense models are scarce. Current adversarial defense methods suffer from reduced accuracy, increased training time, and incomplete defense against adversarial attacks, indicating performance limitations and a lack of robustness. To address these limitations, we propose a composite defense model, the knowledge Distillation and deNoising Network (DiNo-Net), which integrates knowledge distillation and feature denoising techniques. Furthermore, we analyzed a correlation between the loss surface of adversarial perturbations and denoising techniques. Using DiNo-Net, we confirmed that increasing the temperature during the knowledge distillation process effectively amplifies the loss surface around the ground truth. Consequently, this enables more efficient denoising of the adversarial perturbations. It achieved a defense success rate of 72.7%, which is a remarkable improvement over the 41.0% success rate of models with only denoising defense mechanisms. Furthermore, DiNo-Net reduced the training time and maintained higher accuracy, confirming its efficient defense performance. We hope that this relationship will spur the development of fundamental defense strategies. |
| format | Article |
| id | doaj-art-2a06d51d6b6f41d5adf562408aa5a859 |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-2a06d51d6b6f41d5adf562408aa5a8592024-12-13T16:22:01ZengMDPI AGApplied Sciences2076-34172024-11-0114231087210.3390/app142310872Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial DefenseInpyo Hong0Sokjoon Lee1Department of Computer Science, Yonsei University, Seoul 03722, Republic of KoreaDepartment of Computer Engineering, Gachon University, Seongnam 13120, Republic of KoreaEscalating advancements in artificial intelligence (AI) has prompted significant security concerns, especially with its increasing commercialization. This necessitates research on safety measures to securely utilize AI models. Existing AI models are vulnerable to adversarial attacks, which are a specific form of assault methodology. Although various countermeasures have been explored, practical defense models are scarce. Current adversarial defense methods suffer from reduced accuracy, increased training time, and incomplete defense against adversarial attacks, indicating performance limitations and a lack of robustness. To address these limitations, we propose a composite defense model, the knowledge Distillation and deNoising Network (DiNo-Net), which integrates knowledge distillation and feature denoising techniques. Furthermore, we analyzed a correlation between the loss surface of adversarial perturbations and denoising techniques. Using DiNo-Net, we confirmed that increasing the temperature during the knowledge distillation process effectively amplifies the loss surface around the ground truth. Consequently, this enables more efficient denoising of the adversarial perturbations. It achieved a defense success rate of 72.7%, which is a remarkable improvement over the 41.0% success rate of models with only denoising defense mechanisms. Furthermore, DiNo-Net reduced the training time and maintained higher accuracy, confirming its efficient defense performance. We hope that this relationship will spur the development of fundamental defense strategies.https://www.mdpi.com/2076-3417/14/23/10872adversarial attackadversarial robustnessknowledge distillationfeature denoising |
| spellingShingle | Inpyo Hong Sokjoon Lee Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense Applied Sciences adversarial attack adversarial robustness knowledge distillation feature denoising |
| title | Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense |
| title_full | Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense |
| title_fullStr | Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense |
| title_full_unstemmed | Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense |
| title_short | Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense |
| title_sort | exploring synergy of denoising and distillation novel method for efficient adversarial defense |
| topic | adversarial attack adversarial robustness knowledge distillation feature denoising |
| url | https://www.mdpi.com/2076-3417/14/23/10872 |
| work_keys_str_mv | AT inpyohong exploringsynergyofdenoisinganddistillationnovelmethodforefficientadversarialdefense AT sokjoonlee exploringsynergyofdenoisinganddistillationnovelmethodforefficientadversarialdefense |