Learning with noisy labels via clean aware sharpness aware minimization
Abstract Noise label learning has attracted considerable attention owing to its ability to leverage large amounts of inexpensive and imprecise data. Sharpness aware minimization (SAM) has shown effective improvements in the generalization performance in the presence of noisy labels by introducing ad...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-85679-8 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841544805154291712 |
|---|---|
| author | Bin Huang Ying Xie Chaoyang Xu |
| author_facet | Bin Huang Ying Xie Chaoyang Xu |
| author_sort | Bin Huang |
| collection | DOAJ |
| description | Abstract Noise label learning has attracted considerable attention owing to its ability to leverage large amounts of inexpensive and imprecise data. Sharpness aware minimization (SAM) has shown effective improvements in the generalization performance in the presence of noisy labels by introducing adversarial weight perturbations in the model parameter space. However, our experimental observations have shown that the SAM generalization bottleneck primarily stems from the difficulty of finding the correct adversarial perturbation amidst the noisy data. To address this problem, a theoretical analysis of the mismatch in the direction of the parameter perturbation between noise and clean samples during the training process was conducted. Based on these analyses, a clean aware sharpness aware minimization algorithm known as CA-SAM is proposed. CA-SAM dynamically divides the training data into possible likely clean and noisy datasets based on the historical model output and uses likely clean samples to determine the direction of the parameter perturbation. By searching for flat minima in the loss landscape, the objective was to restrict the gradient perturbation direction of noisy samples to align them while preserving the clean samples. By conducting comprehensive experiments and scrutinizing benchmark datasets containing diverse noise patterns and levels, it is demonstrated that our CA-SAM outperforms certain innovative approaches by a substantial margin. |
| format | Article |
| id | doaj-art-2ac63ca0d787424cb9b5c52f4f1ec81c |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-2ac63ca0d787424cb9b5c52f4f1ec81c2025-01-12T12:14:27ZengNature PortfolioScientific Reports2045-23222025-01-0115111210.1038/s41598-025-85679-8Learning with noisy labels via clean aware sharpness aware minimizationBin Huang0Ying Xie1Chaoyang Xu2School of Business, Putian UniversitySchool of Mechanical, Electrical, and Information Engineering, Putian UniversitySchool of Mechanical, Electrical, and Information Engineering, Putian UniversityAbstract Noise label learning has attracted considerable attention owing to its ability to leverage large amounts of inexpensive and imprecise data. Sharpness aware minimization (SAM) has shown effective improvements in the generalization performance in the presence of noisy labels by introducing adversarial weight perturbations in the model parameter space. However, our experimental observations have shown that the SAM generalization bottleneck primarily stems from the difficulty of finding the correct adversarial perturbation amidst the noisy data. To address this problem, a theoretical analysis of the mismatch in the direction of the parameter perturbation between noise and clean samples during the training process was conducted. Based on these analyses, a clean aware sharpness aware minimization algorithm known as CA-SAM is proposed. CA-SAM dynamically divides the training data into possible likely clean and noisy datasets based on the historical model output and uses likely clean samples to determine the direction of the parameter perturbation. By searching for flat minima in the loss landscape, the objective was to restrict the gradient perturbation direction of noisy samples to align them while preserving the clean samples. By conducting comprehensive experiments and scrutinizing benchmark datasets containing diverse noise patterns and levels, it is demonstrated that our CA-SAM outperforms certain innovative approaches by a substantial margin.https://doi.org/10.1038/s41598-025-85679-8Deep neural networksNoisy label learningSharpness aware minimizationModel generalizationLoss landscape |
| spellingShingle | Bin Huang Ying Xie Chaoyang Xu Learning with noisy labels via clean aware sharpness aware minimization Scientific Reports Deep neural networks Noisy label learning Sharpness aware minimization Model generalization Loss landscape |
| title | Learning with noisy labels via clean aware sharpness aware minimization |
| title_full | Learning with noisy labels via clean aware sharpness aware minimization |
| title_fullStr | Learning with noisy labels via clean aware sharpness aware minimization |
| title_full_unstemmed | Learning with noisy labels via clean aware sharpness aware minimization |
| title_short | Learning with noisy labels via clean aware sharpness aware minimization |
| title_sort | learning with noisy labels via clean aware sharpness aware minimization |
| topic | Deep neural networks Noisy label learning Sharpness aware minimization Model generalization Loss landscape |
| url | https://doi.org/10.1038/s41598-025-85679-8 |
| work_keys_str_mv | AT binhuang learningwithnoisylabelsviacleanawaresharpnessawareminimization AT yingxie learningwithnoisylabelsviacleanawaresharpnessawareminimization AT chaoyangxu learningwithnoisylabelsviacleanawaresharpnessawareminimization |