TY - JOUR A1 - Song, Rui A1 - Zhou, Liguo A1 - Lyu, Lingjuan A1 - Festag, Andreas A1 - Knoll, Alois T1 - ResFed: Communication-Efficient Federated Learning With Deep Compressed Residuals JF - IEEE Internet of Things Journal N2 - Federated learning allows for cooperative training among distributed clients by sharing their locally learned model parameters, such as weights or gradients. However, as model size increases, the communication bandwidth required for deployment in wireless networks becomes a bottleneck. To address this, we propose a residual-based federated learning framework (ResFed) that transmits residuals instead of gradients or weights in networks. By predicting model updates at both clients and the server, residuals are calculated as the difference between updated and predicted models and contain more dense information than weights or gradients. We find that the residuals are less sensitive to an increasing compression ratio than other parameters, and hence use lossy compression techniques on residuals to improve communication efficiency for training in federated settings. With the same compression ratio, ResFed outperforms current methods (weight- or gradient-based federated learning) by over 1.4× on federated data sets, including MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100, and FEMNIST, in client-to-server communication, and can also be applied to reduce communication costs for server-to-client communication. UR - https://doi.org/10.1109/JIOT.2023.3324079 Y1 - 2023 UR - https://doi.org/10.1109/JIOT.2023.3324079 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46371 SN - 2327-4662 VL - 11 IS - 6 SP - 9458 EP - 9472 PB - IEEE CY - New York ER - TY - INPR A1 - Song, Rui A1 - Zhou, Liguo A1 - Lyu, Lingjuan A1 - Festag, Andreas A1 - Knoll, Alois T1 - ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals UR - https://doi.org/10.48550/arXiv.2212.05602 Y1 - 2022 UR - https://doi.org/10.48550/arXiv.2212.05602 PB - arXiv CY - Ithaca ER - TY - CHAP A1 - Song, Rui A1 - Zhou, Liguo A1 - Lakshminarasimhan, Venkatnarayanan A1 - Festag, Andreas A1 - Knoll, Alois T1 - Federated Learning Framework Coping with Hierarchical Heterogeneity in Cooperative ITS T2 - 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC) KW - Training KW - Deep learning KW - Data privacy KW - Federated learning KW - Roads KW - Computational modeling KW - Data models Y1 - 2022 SN - 978-1-6654-6880-0 SP - 3502 EP - 3508 PB - IEEE CY - Piscataway ER - TY - JOUR A1 - Zhou, Liguo A1 - Song, Rui A1 - Chen, Guang A1 - Festag, Andreas A1 - Knoll, Alois T1 - Residual encoding framework to compress DNN parameters for fast transfer JF - Knowledge-Based Systems UR - https://doi.org/10.1016/j.knosys.2023.110815 Y1 - 2023 UR - https://doi.org/10.1016/j.knosys.2023.110815 SN - 1872-7409 SN - 0950-7051 VL - 2023 IS - 277 PB - Elsevier CY - Amsterdam ER -