A Flexible Framework for Decentralized Composite Optimization with Compressed Communication
This paper addresses the decentralized composite optimization problem, where a network of agents cooperatively minimize the sum of their local objective functions with non-differentiable terms. We propose a novel communication-efficient decentralized ADMM framework, termed as CE-DADMM, by combining...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Fractal and Fractional |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-3110/8/12/721 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846104548143267840 |
|---|---|
| author | Zhongyi Chang Zhen Zhang Shaofu Yang Jinde Cao |
| author_facet | Zhongyi Chang Zhen Zhang Shaofu Yang Jinde Cao |
| author_sort | Zhongyi Chang |
| collection | DOAJ |
| description | This paper addresses the decentralized composite optimization problem, where a network of agents cooperatively minimize the sum of their local objective functions with non-differentiable terms. We propose a novel communication-efficient decentralized ADMM framework, termed as CE-DADMM, by combining the ADMM framework with the three-point compressed (3PC) communication mechanism. This framework not only covers existing mainstream communication-efficient algorithms but also introduces a series of new algorithms. One of the key features of the CE-DADMM framework is its flexibility, allowing it to adapt to different communication and computation needs, balancing communication efficiency and computational overhead. Notably, when employing quasi-Newton updates, CE-DADMM becomes the first communication-efficient second-order algorithm based on compression that can efficiently handle composite optimization problems. Theoretical analysis shows that, even in the presence of compression errors, the proposed algorithm maintains exact linear convergence when the local objective functions are strongly convex. Finally, numerical experiments demonstrate the algorithm’s impressive communication efficiency. |
| format | Article |
| id | doaj-art-1d983074f736400a98bd16ba0513a4f7 |
| institution | Kabale University |
| issn | 2504-3110 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Fractal and Fractional |
| spelling | doaj-art-1d983074f736400a98bd16ba0513a4f72024-12-27T14:27:07ZengMDPI AGFractal and Fractional2504-31102024-12-0181272110.3390/fractalfract8120721A Flexible Framework for Decentralized Composite Optimization with Compressed CommunicationZhongyi Chang0Zhen Zhang1Shaofu Yang2Jinde Cao3School of Computer Science and Engineering, Southeast University, Nanjing 211189, ChinaSchool of Computer Science and Engineering, Southeast University, Nanjing 211189, ChinaSchool of Computer Science and Engineering, Southeast University, Nanjing 211189, ChinaSchool of Mathematics, Southeast University, Nanjing 211189, ChinaThis paper addresses the decentralized composite optimization problem, where a network of agents cooperatively minimize the sum of their local objective functions with non-differentiable terms. We propose a novel communication-efficient decentralized ADMM framework, termed as CE-DADMM, by combining the ADMM framework with the three-point compressed (3PC) communication mechanism. This framework not only covers existing mainstream communication-efficient algorithms but also introduces a series of new algorithms. One of the key features of the CE-DADMM framework is its flexibility, allowing it to adapt to different communication and computation needs, balancing communication efficiency and computational overhead. Notably, when employing quasi-Newton updates, CE-DADMM becomes the first communication-efficient second-order algorithm based on compression that can efficiently handle composite optimization problems. Theoretical analysis shows that, even in the presence of compression errors, the proposed algorithm maintains exact linear convergence when the local objective functions are strongly convex. Finally, numerical experiments demonstrate the algorithm’s impressive communication efficiency.https://www.mdpi.com/2504-3110/8/12/721decentralized composite optimizationADMMquasi-Newtoncommunication-efficient mechanism |
| spellingShingle | Zhongyi Chang Zhen Zhang Shaofu Yang Jinde Cao A Flexible Framework for Decentralized Composite Optimization with Compressed Communication Fractal and Fractional decentralized composite optimization ADMM quasi-Newton communication-efficient mechanism |
| title | A Flexible Framework for Decentralized Composite Optimization with Compressed Communication |
| title_full | A Flexible Framework for Decentralized Composite Optimization with Compressed Communication |
| title_fullStr | A Flexible Framework for Decentralized Composite Optimization with Compressed Communication |
| title_full_unstemmed | A Flexible Framework for Decentralized Composite Optimization with Compressed Communication |
| title_short | A Flexible Framework for Decentralized Composite Optimization with Compressed Communication |
| title_sort | flexible framework for decentralized composite optimization with compressed communication |
| topic | decentralized composite optimization ADMM quasi-Newton communication-efficient mechanism |
| url | https://www.mdpi.com/2504-3110/8/12/721 |
| work_keys_str_mv | AT zhongyichang aflexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication AT zhenzhang aflexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication AT shaofuyang aflexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication AT jindecao aflexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication AT zhongyichang flexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication AT zhenzhang flexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication AT shaofuyang flexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication AT jindecao flexibleframeworkfordecentralizedcompositeoptimizationwithcompressedcommunication |