Asynchronous Federated Learning Through Online Linear Regressions
In the practical scenario of Federated Learning (FL), clients upload their local model to a server at different times owing to heterogeneity in the clients’ device environment. Therefore, Asynchronous Federated Learning (AFL) has been aggressively studied recently. Although the initial mo...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10811896/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846102112433340416 |
|---|---|
| author | Taiga Kashima Ayako Amma Hideki Nakayama |
| author_facet | Taiga Kashima Ayako Amma Hideki Nakayama |
| author_sort | Taiga Kashima |
| collection | DOAJ |
| description | In the practical scenario of Federated Learning (FL), clients upload their local model to a server at different times owing to heterogeneity in the clients’ device environment. Therefore, Asynchronous Federated Learning (AFL) has been aggressively studied recently. Although the initial motivation for AFL is to reduce the difficulties of FL, AFL itself also has problems in practice (that is, the latest uploaded local model affects the performance of the global model). As a scientific challenge, we would like to develop a simple yet broadly applicable concept for AFL; thus, we revisited a classic machine learning theory and found a high conceptual affinity between AFL and online linear regression—both are problem formulations for sequential inputs. On the basis of this underlying philosophy, we propose a framework for AFL in which classic online linear regression techniques are utilized in server aggregation to realize an asynchronous global model update. Our framework can provide a convergence guarantee of the global model performance relative to a synchronized aggregation method through the theoretical guarantee in online linear regressions. The experiments show the superiority of our asynchronous updates in a server in both image and text tasks under non-IID data. |
| format | Article |
| id | doaj-art-1b6f4a109c66479eadda50e27dfd24d4 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-1b6f4a109c66479eadda50e27dfd24d42024-12-28T00:00:52ZengIEEEIEEE Access2169-35362024-01-011219513119514410.1109/ACCESS.2024.352100910811896Asynchronous Federated Learning Through Online Linear RegressionsTaiga Kashima0https://orcid.org/0000-0002-5709-4752Ayako Amma1Hideki Nakayama2Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, JapanWoven by Toyota Inc., Tokyo, JapanGraduate School of Information Science and Technology, The University of Tokyo, Tokyo, JapanIn the practical scenario of Federated Learning (FL), clients upload their local model to a server at different times owing to heterogeneity in the clients’ device environment. Therefore, Asynchronous Federated Learning (AFL) has been aggressively studied recently. Although the initial motivation for AFL is to reduce the difficulties of FL, AFL itself also has problems in practice (that is, the latest uploaded local model affects the performance of the global model). As a scientific challenge, we would like to develop a simple yet broadly applicable concept for AFL; thus, we revisited a classic machine learning theory and found a high conceptual affinity between AFL and online linear regression—both are problem formulations for sequential inputs. On the basis of this underlying philosophy, we propose a framework for AFL in which classic online linear regression techniques are utilized in server aggregation to realize an asynchronous global model update. Our framework can provide a convergence guarantee of the global model performance relative to a synchronized aggregation method through the theoretical guarantee in online linear regressions. The experiments show the superiority of our asynchronous updates in a server in both image and text tasks under non-IID data.https://ieeexplore.ieee.org/document/10811896/Asynchronous federated learningonline linear regressionfederated learning |
| spellingShingle | Taiga Kashima Ayako Amma Hideki Nakayama Asynchronous Federated Learning Through Online Linear Regressions IEEE Access Asynchronous federated learning online linear regression federated learning |
| title | Asynchronous Federated Learning Through Online Linear Regressions |
| title_full | Asynchronous Federated Learning Through Online Linear Regressions |
| title_fullStr | Asynchronous Federated Learning Through Online Linear Regressions |
| title_full_unstemmed | Asynchronous Federated Learning Through Online Linear Regressions |
| title_short | Asynchronous Federated Learning Through Online Linear Regressions |
| title_sort | asynchronous federated learning through online linear regressions |
| topic | Asynchronous federated learning online linear regression federated learning |
| url | https://ieeexplore.ieee.org/document/10811896/ |
| work_keys_str_mv | AT taigakashima asynchronousfederatedlearningthroughonlinelinearregressions AT ayakoamma asynchronousfederatedlearningthroughonlinelinearregressions AT hidekinakayama asynchronousfederatedlearningthroughonlinelinearregressions |