The search result changed since you submitted your search request. Documents might be displayed in a different sort order.
  • search hit 17 of 112
Back to Result List

ResFed: Communication-Efficient Federated Learning With Deep Compressed Residuals

  • Federated learning allows for cooperative training among distributed clients by sharing their locally learned model parameters, such as weights or gradients. However, as model size increases, the communication bandwidth required for deployment in wireless networks becomes a bottleneck. To address this, we propose a residual-based federated learning framework (ResFed) that transmits residuals instead of gradients or weights in networks. By predicting model updates at both clients and the server, residuals are calculated as the difference between updated and predicted models and contain more dense information than weights or gradients. We find that the residuals are less sensitive to an increasing compression ratio than other parameters, and hence use lossy compression techniques on residuals to improve communication efficiency for training in federated settings. With the same compression ratio, ResFed outperforms current methods (weight- or gradient-based federated learning) by over 1.4× on federated data sets, including MNIST,Federated learning allows for cooperative training among distributed clients by sharing their locally learned model parameters, such as weights or gradients. However, as model size increases, the communication bandwidth required for deployment in wireless networks becomes a bottleneck. To address this, we propose a residual-based federated learning framework (ResFed) that transmits residuals instead of gradients or weights in networks. By predicting model updates at both clients and the server, residuals are calculated as the difference between updated and predicted models and contain more dense information than weights or gradients. We find that the residuals are less sensitive to an increasing compression ratio than other parameters, and hence use lossy compression techniques on residuals to improve communication efficiency for training in federated settings. With the same compression ratio, ResFed outperforms current methods (weight- or gradient-based federated learning) by over 1.4× on federated data sets, including MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100, and FEMNIST, in client-to-server communication, and can also be applied to reduce communication costs for server-to-client communication.show moreshow less

Download full text files

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Rui SongORCiD, Liguo ZhouORCiD, Lingjuan Lyu, Andreas FestagORCiD, Alois KnollORCiD
Language:English
Document Type:Article
Year of first Publication:2023
published in (English):IEEE Internet of Things Journal
Publisher:IEEE
Place of publication:New York
ISSN:2327-4662
Volume:11
Issue:6
First Page:9458
Last Page:9472
Review:peer-review
Open Access:ja
Version:published
URN:urn:nbn:de:bvb:573-46371
Related Identifier:https://doi.org/10.1109/JIOT.2023.3324079
Faculties / Institutes / Organizations:Fakultät Elektro- und Informationstechnik
CARISSMA Institute of Electric, Connected and Secure Mobility (C-ECOS)
Fraunhofer-Anwendungszentrum "Vernetzte Mobilität und Infrastruktur"
Licence (German):License Logo Creative Commons BY 4.0
Release Date:2024/03/25