SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight Optimization
Federated Learning (FL) enables decentralized training of machine learning models across multiple clients, preserving data privacy by aggregating locally trained models without sharing raw data. Traditional aggregation methods, such as Federated Averaging (FedAvg), often assume uniform client contri...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Open Journal of the Communications Society |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10786254/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846118260900102144 |
|---|---|
| author | Ayoub El-Niss Ahmad Alzu'Bi Abdelrahman Abuarqoub Mohammad Hammoudeh Ammar Muthanna |
| author_facet | Ayoub El-Niss Ahmad Alzu'Bi Abdelrahman Abuarqoub Mohammad Hammoudeh Ammar Muthanna |
| author_sort | Ayoub El-Niss |
| collection | DOAJ |
| description | Federated Learning (FL) enables decentralized training of machine learning models across multiple clients, preserving data privacy by aggregating locally trained models without sharing raw data. Traditional aggregation methods, such as Federated Averaging (FedAvg), often assume uniform client contributions, leading to suboptimal global models in heterogeneous data environments. This article introduces SimProx, a novel FL approach for aggregation that addresses heterogeneity in data through three key improvements. First, SimProx employs a composite similarity-based weighting mechanism, integrating cosine and Gaussian similarity measures to dynamically optimize client contributions. Then, it incorporates a proximal term in the client weighting scheme, using gradient norms to prioritize updates closer to the global optimum, thereby enhancing model convergence and robustness. Finally, a dynamic parameter learning technique is introduced, which adapts the balance between similarity measures based on data heterogeneity, refining the aggregation process. Extensive experiments on standard benchmarking datasets and real-world multimodal data demonstrate that SimProx significantly outperforms traditional methods like FedAvg in terms of accuracy. SimProx offers a scalable and effective solution for decentralized deep learning in diverse and heterogeneous environments. |
| format | Article |
| id | doaj-art-2f8978dc46b147f68b460c6c7cc0b9cb |
| institution | Kabale University |
| issn | 2644-125X |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of the Communications Society |
| spelling | doaj-art-2f8978dc46b147f68b460c6c7cc0b9cb2024-12-18T00:03:12ZengIEEEIEEE Open Journal of the Communications Society2644-125X2024-01-0157806781710.1109/OJCOMS.2024.351381610786254SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight OptimizationAyoub El-Niss0https://orcid.org/0009-0009-3776-3734Ahmad Alzu'Bi1https://orcid.org/0000-0001-5466-0379Abdelrahman Abuarqoub2https://orcid.org/0000-0001-6576-8932Mohammad Hammoudeh3https://orcid.org/0000-0003-1058-0996Ammar Muthanna4https://orcid.org/0000-0003-0213-8145Department of Computer Science, Jordan University of Science and Technology, Irbid, JordanDepartment of Computer Science, Jordan University of Science and Technology, Irbid, JordanDepartment of Applied Computing, Cardiff School of Technologies, Cardiff, U.K.Department of Computing and Mathematics, Manchester Metropolitan University, Manchester, U.K.Department of Telecommunication Networks and Data Transmission, St. Petersburg State University of Telecommunication, Saint Petersburg, RussiaFederated Learning (FL) enables decentralized training of machine learning models across multiple clients, preserving data privacy by aggregating locally trained models without sharing raw data. Traditional aggregation methods, such as Federated Averaging (FedAvg), often assume uniform client contributions, leading to suboptimal global models in heterogeneous data environments. This article introduces SimProx, a novel FL approach for aggregation that addresses heterogeneity in data through three key improvements. First, SimProx employs a composite similarity-based weighting mechanism, integrating cosine and Gaussian similarity measures to dynamically optimize client contributions. Then, it incorporates a proximal term in the client weighting scheme, using gradient norms to prioritize updates closer to the global optimum, thereby enhancing model convergence and robustness. Finally, a dynamic parameter learning technique is introduced, which adapts the balance between similarity measures based on data heterogeneity, refining the aggregation process. Extensive experiments on standard benchmarking datasets and real-world multimodal data demonstrate that SimProx significantly outperforms traditional methods like FedAvg in terms of accuracy. SimProx offers a scalable and effective solution for decentralized deep learning in diverse and heterogeneous environments.https://ieeexplore.ieee.org/document/10786254/Federated learningdecentralized networkweighted aggregationdata heterogeneitydeep learningmultimodal classification |
| spellingShingle | Ayoub El-Niss Ahmad Alzu'Bi Abdelrahman Abuarqoub Mohammad Hammoudeh Ammar Muthanna SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight Optimization IEEE Open Journal of the Communications Society Federated learning decentralized network weighted aggregation data heterogeneity deep learning multimodal classification |
| title | SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight Optimization |
| title_full | SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight Optimization |
| title_fullStr | SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight Optimization |
| title_full_unstemmed | SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight Optimization |
| title_short | SimProx: A Similarity-Based Aggregation in Federated Learning With Client Weight Optimization |
| title_sort | simprox a similarity based aggregation in federated learning with client weight optimization |
| topic | Federated learning decentralized network weighted aggregation data heterogeneity deep learning multimodal classification |
| url | https://ieeexplore.ieee.org/document/10786254/ |
| work_keys_str_mv | AT ayoubelniss simproxasimilaritybasedaggregationinfederatedlearningwithclientweightoptimization AT ahmadalzubi simproxasimilaritybasedaggregationinfederatedlearningwithclientweightoptimization AT abdelrahmanabuarqoub simproxasimilaritybasedaggregationinfederatedlearningwithclientweightoptimization AT mohammadhammoudeh simproxasimilaritybasedaggregationinfederatedlearningwithclientweightoptimization AT ammarmuthanna simproxasimilaritybasedaggregationinfederatedlearningwithclientweightoptimization |