Visual explanation method for reversible neural networks
The issue of model explainability has gained significant attention in understanding the vulnerabilities and anonymous decision-making processes inherent in deep neural networks (DNN).While there has been considerable research on explainability for traditional DNN, there is a lack of exploration on t...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
POSTS&TELECOM PRESS Co., LTD
2023-12-01
|
| Series: | 网络与信息安全学报 |
| Subjects: | |
| Online Access: | http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2023090 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841529595012055040 |
|---|---|
| author | Xinying MU Bingbing SONG Fanxiao LI Yisen ZHENG Wei ZHOU Yunyun DONG |
| author_facet | Xinying MU Bingbing SONG Fanxiao LI Yisen ZHENG Wei ZHOU Yunyun DONG |
| author_sort | Xinying MU |
| collection | DOAJ |
| description | The issue of model explainability has gained significant attention in understanding the vulnerabilities and anonymous decision-making processes inherent in deep neural networks (DNN).While there has been considerable research on explainability for traditional DNN, there is a lack of exploration on the operation mechanism and explainability of reversible neural networks (RevNN).Additionally, the existing explanation methods for traditional DNN are not suitable for RevNN and suffer from issues such as excessive noise and gradient saturation.To address these limitations, a visual explanation method called visual explanation method for reversible neural network (VERN) was proposed for RevNN.VERN leverages the reversible property of RevNN and is based on the class-activation mapping mechanism.The correspondence between the feature map and the input image was explored by VERN, allowing for the mapping of classification weights of regional feature maps to the corresponding regions of the input image.The importance of each region for model decision-making was revealed through this process, which generates a basis for model decision-making.Experimental comparisons with other explanation methods on generalized datasets demonstrate that VERN achieves a more focused visual effect, surpassing suboptimal methods with up to 7.80% improvement in average drop (AD) metrics and up to 6.05% improvement in average increase (AI) metrics in recognition tasks.VERN also exhibits an 82.00% level of localization for the maximum point of the heat value.Furthermore, VERN can be applied to explain traditional DNN and exhibits good scalability, improving the performance of other methods in explaining RevNN.Furthermore, through adversarial attack analysis experiments, it is observed that adversarial attacks alter the decision basis of the model.This is reflected in the misalignment of the model’s attention regions, thereby aiding in the exploration of the operation mechanism of adversarial attacks. |
| format | Article |
| id | doaj-art-29b00d6e844d4c7890d200c7ddfff1aa |
| institution | Kabale University |
| issn | 2096-109X |
| language | English |
| publishDate | 2023-12-01 |
| publisher | POSTS&TELECOM PRESS Co., LTD |
| record_format | Article |
| series | 网络与信息安全学报 |
| spelling | doaj-art-29b00d6e844d4c7890d200c7ddfff1aa2025-01-15T03:16:55ZengPOSTS&TELECOM PRESS Co., LTD网络与信息安全学报2096-109X2023-12-01915416559580856Visual explanation method for reversible neural networksXinying MUBingbing SONGFanxiao LIYisen ZHENGWei ZHOUYunyun DONGThe issue of model explainability has gained significant attention in understanding the vulnerabilities and anonymous decision-making processes inherent in deep neural networks (DNN).While there has been considerable research on explainability for traditional DNN, there is a lack of exploration on the operation mechanism and explainability of reversible neural networks (RevNN).Additionally, the existing explanation methods for traditional DNN are not suitable for RevNN and suffer from issues such as excessive noise and gradient saturation.To address these limitations, a visual explanation method called visual explanation method for reversible neural network (VERN) was proposed for RevNN.VERN leverages the reversible property of RevNN and is based on the class-activation mapping mechanism.The correspondence between the feature map and the input image was explored by VERN, allowing for the mapping of classification weights of regional feature maps to the corresponding regions of the input image.The importance of each region for model decision-making was revealed through this process, which generates a basis for model decision-making.Experimental comparisons with other explanation methods on generalized datasets demonstrate that VERN achieves a more focused visual effect, surpassing suboptimal methods with up to 7.80% improvement in average drop (AD) metrics and up to 6.05% improvement in average increase (AI) metrics in recognition tasks.VERN also exhibits an 82.00% level of localization for the maximum point of the heat value.Furthermore, VERN can be applied to explain traditional DNN and exhibits good scalability, improving the performance of other methods in explaining RevNN.Furthermore, through adversarial attack analysis experiments, it is observed that adversarial attacks alter the decision basis of the model.This is reflected in the misalignment of the model’s attention regions, thereby aiding in the exploration of the operation mechanism of adversarial attacks.http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2023090model explainabilityreversible neural networkvisualizationclass activation mappingartificial intelligence security |
| spellingShingle | Xinying MU Bingbing SONG Fanxiao LI Yisen ZHENG Wei ZHOU Yunyun DONG Visual explanation method for reversible neural networks 网络与信息安全学报 model explainability reversible neural network visualization class activation mapping artificial intelligence security |
| title | Visual explanation method for reversible neural networks |
| title_full | Visual explanation method for reversible neural networks |
| title_fullStr | Visual explanation method for reversible neural networks |
| title_full_unstemmed | Visual explanation method for reversible neural networks |
| title_short | Visual explanation method for reversible neural networks |
| title_sort | visual explanation method for reversible neural networks |
| topic | model explainability reversible neural network visualization class activation mapping artificial intelligence security |
| url | http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2023090 |
| work_keys_str_mv | AT xinyingmu visualexplanationmethodforreversibleneuralnetworks AT bingbingsong visualexplanationmethodforreversibleneuralnetworks AT fanxiaoli visualexplanationmethodforreversibleneuralnetworks AT yisenzheng visualexplanationmethodforreversibleneuralnetworks AT weizhou visualexplanationmethodforreversibleneuralnetworks AT yunyundong visualexplanationmethodforreversibleneuralnetworks |