TY - INPR A1 - Song, Rui A1 - Liu, Dai A1 - Chen, Dave Zhenyu A1 - Festag, Andreas A1 - Trinitis, Carsten A1 - Schulz, Martin A1 - Knoll, Alois T1 - Federated Learning via Decentralized Dataset Distillation in Resource Constrained Edge Environments N2 - In federated learning, all networked clients contribute to the model training cooperatively. However, with model sizes increasing, even sharing the trained partial models often leads to severe communication bottlenecks in underlying networks, especially when communicated iteratively. In this paper, we introduce a federated learning framework FedD3 requiring only one-shot communication by integrating dataset distillation instances. Instead of sharing model updates in other federated learning approaches, FedD3 allows the connected clients to distill the local datasets independently, and then aggregates those decentralized distilled datasets (e.g. a few unrecognizable images) from networks for model training. Our experimental results show that FedD3 significantly outperforms other federated learning frameworks in terms of needed communication volumes, while it provides the additional benefit to be able to balance the trade-off between accuracy and communication cost, depending on usage scenario or target dataset. For instance, for training an AlexNet model on CIFAR-10 with 10 clients under non-independent and identically distributed (Non-IID) setting, FedD3 can either increase the accuracy by over 71% with a similar communication volume, or save 98% of communication volume, while reaching the same accuracy, compared to other one-shot federated learning approaches. UR - https://doi.org/10.48550/arXiv.2208.11311 Y1 - 2022 UR - https://doi.org/10.48550/arXiv.2208.11311 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-60054 PB - arXiv CY - Ithaca ER - TY - INPR A1 - Song, Rui A1 - Liang, Chenwei A1 - Xia, Yan A1 - Zimmer, Walter A1 - Cao, Hu A1 - Caesar, Holger A1 - Festag, Andreas A1 - Knoll, Alois T1 - CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Driving N2 - Dynamic scene rendering opens new avenues in autonomous driving by enabling closed-loop simulations with photorealistic data, which is crucial for validating end-to-end algorithms. However, the complex and highly dynamic nature of traffic environments presents significant challenges in accurately rendering these scenes. In this paper, we introduce a novel 4D Gaussian Splatting (4DGS) approach, which incorporates context and temporal deformation awareness to improve dynamic scene rendering. Specifically, we employ a 2D semantic segmentation foundation model to self-supervise the 4D semantic features of Gaussians, ensuring meaningful contextual embedding. Simultaneously, we track the temporal deformation of each Gaussian across adjacent frames. By aggregating and encoding both semantic and temporal deformation features, each Gaussian is equipped with cues for potential deformation compensation within 3D space, facilitating a more precise representation of dynamic scenes. Experimental results show that our method improves 4DGS's ability to capture fine details in dynamic scene rendering for autonomous driving and outperforms other self-supervised methods in 4D reconstruction and novel view synthesis. Furthermore, CoDa-4DGS deforms semantic features with each Gaussian, enabling broader applications. UR - https://doi.org/10.48550/arXiv.2503.06744 Y1 - 2025 UR - https://doi.org/10.48550/arXiv.2503.06744 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-58313 PB - arXiv CY - Ithaca ER - TY - CHAP A1 - Song, Rui A1 - Festag, Andreas A1 - Jagtap, Abhishek Dinkar A1 - Bialdyga, Maximilian A1 - Yan, Zhiran A1 - Otte, Maximilian A1 - Sadashivaiah, Sanath Tiptur A1 - Knoll, Alois T1 - First Mile: An Open Innovation Lab for Infrastructure-Assisted Cooperative Intelligent Transportation Systems T2 - 2024 IEEE Intelligent Vehicles Symposium (IV) UR - https://doi.org/10.1109/IV55156.2024.10588500 Y1 - 2024 UR - https://doi.org/10.1109/IV55156.2024.10588500 SN - 979-8-3503-4881-1 SP - 1635 EP - 1642 PB - IEEE CY - Piscataway ER - TY - CHAP A1 - Song, Rui A1 - Liu, Dai A1 - Chen, Dave Zhenyu A1 - Festag, Andreas A1 - Trinitis, Carsten A1 - Schulz, Martin A1 - Knoll, Alois T1 - Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments T2 - IJCNN 2023 Conference Proceedings UR - https://doi.org/10.1109/IJCNN54540.2023.10191879 Y1 - 2023 UR - https://doi.org/10.1109/IJCNN54540.2023.10191879 SN - 978-1-6654-8867-9 PB - IEEE CY - Piscataway ER - TY - CHAP A1 - Agrawal, Shiva A1 - Song, Rui A1 - Doycheva, Kristina A1 - Knoll, Alois A1 - Elger, Gordon ED - Klein, Cornel ED - Jarke, Matthias ED - Ploeg, Jeroen ED - Helfert, Markus ED - Berns, Karsten ED - Gusikhin, Oleg T1 - Intelligent Roadside Infrastructure for Connected Mobility T2 - Smart Cities, Green Technologies, and Intelligent Transport Systems: 11th International Conference, SMARTGREENS 2022 and 8th International Conference, VEHITS 2022: Revised Selected Papers UR - https://doi.org/10.1007/978-3-031-37470-8_6 Y1 - 2023 UR - https://doi.org/10.1007/978-3-031-37470-8_6 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-41761 SN - 978-3-031-37470-8 SN - 1865-0937 SP - 134 EP - 157 PB - Springer CY - Cham ER - TY - JOUR A1 - Song, Rui A1 - Zhou, Liguo A1 - Lyu, Lingjuan A1 - Festag, Andreas A1 - Knoll, Alois T1 - ResFed: Communication-Efficient Federated Learning With Deep Compressed Residuals JF - IEEE Internet of Things Journal N2 - Federated learning allows for cooperative training among distributed clients by sharing their locally learned model parameters, such as weights or gradients. However, as model size increases, the communication bandwidth required for deployment in wireless networks becomes a bottleneck. To address this, we propose a residual-based federated learning framework (ResFed) that transmits residuals instead of gradients or weights in networks. By predicting model updates at both clients and the server, residuals are calculated as the difference between updated and predicted models and contain more dense information than weights or gradients. We find that the residuals are less sensitive to an increasing compression ratio than other parameters, and hence use lossy compression techniques on residuals to improve communication efficiency for training in federated settings. With the same compression ratio, ResFed outperforms current methods (weight- or gradient-based federated learning) by over 1.4× on federated data sets, including MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100, and FEMNIST, in client-to-server communication, and can also be applied to reduce communication costs for server-to-client communication. UR - https://doi.org/10.1109/JIOT.2023.3324079 Y1 - 2023 UR - https://doi.org/10.1109/JIOT.2023.3324079 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46371 SN - 2327-4662 VL - 11 IS - 6 SP - 9458 EP - 9472 PB - IEEE CY - New York ER - TY - JOUR A1 - Hegde, Anupama A1 - Song, Rui A1 - Festag, Andreas T1 - Radio Resource Allocation in 5G-NR V2X: A Multi-Agent Actor-Critic Based Approach JF - IEEE Access N2 - The efficiency of radio resource allocation and scheduling procedures in Cellular Vehicle-to-X (Cellular V2X) communication networks directly affects link quality in terms of latency and reliability. However, owing to the continuous movement of vehicles, it is impossible to have a centralized coordinating unit at all times to manage the allocation of radio resources. In the unmanaged mode of the fifth generation new radio (5G-NR) V2X, the sensing-based semi-persistent scheduling (SB-SPS) loses its effectiveness when V2X data messages become aperiodic with varying data sizes. This leads to misinformed resource allocation decisions among vehicles and frequent resource collisions. To improve resource selection, this study formulates the Cellular V2X communication network as a decentralized multi-agent networked markov decision process (MDP) where each vehicle agent executes an actor-critic-based radio resource scheduler. Developing further the actor-critic methodology for the radio resource allocation problem in Cellular V2X, two variants are derived: independent actor-critic (IAC) and shared experience actor-critic (SEAC). Results from simulation studies indicate that the actor-critic schedulers improve reliability, achieving a 15-20% higher probability of reception under high vehicular density scenarios with aperiodic traffic patterns. UR - https://doi.org/10.1109/ACCESS.2023.3305267 KW - Cellular V2X KW - radio resource allocation KW - deep reinforcement learning Y1 - 2023 UR - https://doi.org/10.1109/ACCESS.2023.3305267 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-39352 SN - 2169-3536 VL - 11 SP - 87225 EP - 87244 PB - IEEE CY - New York ER - TY - INPR A1 - Song, Rui A1 - Zhou, Liguo A1 - Lyu, Lingjuan A1 - Festag, Andreas A1 - Knoll, Alois T1 - ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals UR - https://doi.org/10.48550/arXiv.2212.05602 Y1 - 2022 UR - https://doi.org/10.48550/arXiv.2212.05602 PB - arXiv CY - Ithaca ER - TY - CHAP A1 - Festag, Andreas A1 - Song, Rui T1 - Analysis of Existing Approaches for Information Sharing in Cooperative Intelligent Transport Systems BT - SENSORIS and V2X Messaging T2 - FISITA World Congress 2021 Y1 - 2021 UR - https://www.fisita.com/library/f2020-acm-012 N1 - Schreibfehler im Titel auf Website der FISITA Library: "Analysis of Existing Approaches for Information Sharingin Cooperative Intelligent Transport Systems" PB - FISITA CY - Bishops Stortford ER - TY - CHAP A1 - Song, Rui A1 - Hegde, Anupama A1 - Senel, Numan A1 - Knoll, Alois A1 - Festag, Andreas T1 - Edge-Aided Sensor Data Sharing in Vehicular Communication Networks T2 - 2022 IEEE 95th Vehicular Technology Conference: (VTC2022-Spring) Proceedings UR - https://doi.org/10.1109/VTC2022-Spring54318.2022.9860849 KW - sensor data sharing and fusion KW - cooperative perception KW - vehicular communication KW - distributed estimation KW - measurement noise Y1 - 2022 UR - https://doi.org/10.1109/VTC2022-Spring54318.2022.9860849 SN - 978-1-6654-8243-1 SN - 2577-2465 PB - IEEE CY - Piscataway (NJ) ER - TY - CHAP A1 - Song, Rui A1 - Zhou, Liguo A1 - Lakshminarasimhan, Venkatnarayanan A1 - Festag, Andreas A1 - Knoll, Alois T1 - Federated Learning Framework Coping with Hierarchical Heterogeneity in Cooperative ITS T2 - 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC) KW - Training KW - Deep learning KW - Data privacy KW - Federated learning KW - Roads KW - Computational modeling KW - Data models Y1 - 2022 SN - 978-1-6654-6880-0 SP - 3502 EP - 3508 PB - IEEE CY - Piscataway ER - TY - JOUR A1 - Zhou, Liguo A1 - Song, Rui A1 - Chen, Guang A1 - Festag, Andreas A1 - Knoll, Alois T1 - Residual encoding framework to compress DNN parameters for fast transfer JF - Knowledge-Based Systems UR - https://doi.org/10.1016/j.knosys.2023.110815 Y1 - 2023 UR - https://doi.org/10.1016/j.knosys.2023.110815 SN - 1872-7409 SN - 0950-7051 VL - 2023 IS - 277 PB - Elsevier CY - Amsterdam ER - TY - CHAP A1 - Song, Rui A1 - Lyu, Lingjuan A1 - Jiang, Wei A1 - Festag, Andreas A1 - Knoll, Alois T1 - V2X-Boosted Federated Learning for Cooperative Intelligent Transportation Systems with Contextual Client Selection T2 - CoPerception: Collaborative Perception and Learning, ICRA 2023 workshop Y1 - 2023 UR - https://drive.google.com/file/d/1-OQAeNryd7-lksCTuWnhhyydpFSnw5On/view?usp=share_link ER - TY - INPR A1 - Song, Rui A1 - Liang, Chenwei A1 - Cao, Hu A1 - Yan, Zhiran A1 - Zimmer, Walter A1 - Gross, Markus A1 - Festag, Andreas A1 - Knoll, Alois T1 - Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles N2 - Collaborative perception in automated vehicles leverages the exchange of information between agents, aiming to elevate perception results. Previous camera-based collaborative 3D perception methods typically employ 3D bounding boxes or bird's eye views as representations of the environment. However, these approaches fall short in offering a comprehensive 3D environmental prediction. To bridge this gap, we introduce the first method for collaborative 3D semantic occupancy prediction. Particularly, it improves local 3D semantic occupancy predictions by hybrid fusion of (i) semantic and occupancy task features, and (ii) compressed orthogonal attention features shared between vehicles. Additionally, due to the lack of a collaborative perception dataset designed for semantic occupancy prediction, we augment a current collaborative perception dataset to include 3D collaborative semantic occupancy labels for a more robust evaluation. The experimental findings highlight that: (i) our collaborative semantic occupancy predictions excel above the results from single vehicles by over 30%, and (ii) models anchored on semantic occupancy outpace state-of-the-art collaborative 3D detection techniques in subsequent perception applications, showcasing enhanced accuracy and enriched semantic-awareness in road environments. UR - https://doi.org/10.48550/arXiv.2402.07635 Y1 - 2024 UR - https://doi.org/10.48550/arXiv.2402.07635 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46035 PB - arXiv CY - Ithaca ER - TY - JOUR A1 - Song, Rui A1 - Xu, Runsheng A1 - Festag, Andreas A1 - Ma, Jiaqi A1 - Knoll, Alois T1 - FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems JF - IEEE Transactions on Intelligent Vehicles N2 - Bird's eye view (BEV) perception is becoming increasingly important in the field of autonomous driving. It uses multi-view camera data to learn a transformer model that directly projects the perception of the road environment onto the BEV perspective. However, training a transformer model often requires a large amount of data, and as camera data for road traffic are often private, they are typically not shared. Federated learning offers a solution that enables clients to collaborate and train models without exchanging data but model parameters. In this paper, we introduce FedBEVT, a federated transformer learning approach for BEV perception. In order to address two common data heterogeneity issues in FedBEVT: (i) diverse sensor poses, and (ii) varying sensor numbers in perception systems, we propose two approaches - Federated Learning with Camera-Attentive Personalization (FedCaP) and Adaptive Multi-Camera Masking (AMCM), respectively. To evaluate our method in real-world settings, we create a dataset consisting of four typical federated use cases. Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving. UR - https://doi.org/10.1109/TIV.2023.3310674 Y1 - 2023 UR - https://doi.org/10.1109/TIV.2023.3310674 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-42789 SN - 2379-8904 SN - 2379-8858 VL - 9 IS - 1 SP - 958 EP - 969 PB - IEEE CY - New York ER - TY - CHAP A1 - Agrawal, Shiva A1 - Song, Rui A1 - Kohli, Akhil A1 - Korb, Andreas A1 - Andre, Maximilian A1 - Holzinger, Erik A1 - Elger, Gordon ED - Ploeg, Jeroen ED - Helfert, Markus ED - Berns, Karsten ED - Gusikhin, Oleg T1 - Concept of Smart Infrastructure for Connected Vehicle Assist and Traffic Flow Optimization T2 - Proceedings of the 8th International Conference on Vehicle Technology and Intelligent Transport Systems N2 - The smart infrastructure units can play a vital role to develop smart cities of the future and in assisting automated vehicles on the road by providing extended perception and timely warnings to avoid accidents. This paper focuses on the development of such an infrastructure unit, that is specifically designed for a pedestrian crossing junction. It can control traffic lights at the junction by real-time environment perception through its sensors and can optimize the flow of vehicles and passing vulnerable road users (VRUs). Moreover, it can assist on-road vehicles by providing real-time information and critical warnings via a v2x module. This paper further describes different use-cases of the work, all major hardware components involved in the development of smart infrastructure unit, referred to as an edge, different sensor fusion approaches using the camera, radar, and lidar mounted on the edge for environment perception, various modes of communication including v2x, system design UR - https://doi.org/10.5220/0011068800003191 KW - Road Side Unit (RSU) KW - Smart Infrastructure KW - Radar KW - LiDAR KW - Camera KW - Perception KW - Sensor Data Fusion KW - Vehicle-to-everything (V2X) KW - Communication KW - Simulation KW - ITS Y1 - 2022 UR - https://doi.org/10.5220/0011068800003191 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-30926 SN - 978-989-758-573-9 SN - 2184-495X SP - 360 EP - 367 PB - SciTePress CY - Setúbal ER - TY - INPR A1 - Song, Rui A1 - Lyu, Lingjuan A1 - Jiang, Wei A1 - Festag, Andreas A1 - Knoll, Alois T1 - V2X-Boosted Federated Learning for Cooperative Intelligent Transportation Systems with Contextual Client Selection N2 - Machine learning (ML) has revolutionized transportation systems, enabling autonomous driving and smart traffic services. Federated learning (FL) overcomes privacy constraints by training ML models in distributed systems, exchanging model parameters instead of raw data. However, the dynamic states of connected vehicles affect the network connection quality and influence the FL performance. To tackle this challenge, we propose a contextual client selection pipeline that uses Vehicle-to-Everything (V2X) messages to select clients based on the predicted communication latency. The pipeline includes: (i) fusing V2X messages, (ii) predicting future traffic topology, (iii) pre-clustering clients based on local data distribution similarity, and (iv) selecting clients with minimal latency for future model aggregation. Experiments show that our pipeline outperforms baselines on various datasets, particularly in non-iid settings. UR - https://doi.org/10.48550/arXiv.2305.11654 Y1 - 2023 UR - https://doi.org/10.48550/arXiv.2305.11654 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-41441 PB - arXiv CY - Ithaca ER - TY - CHAP A1 - Jagtap, Abhishek Dinkar A1 - Song, Rui A1 - Sadashivaiah, Sanath Tiptur A1 - Festag, Andreas T1 - V2X-Gaussians: Gaussian Splatting for Multi-Agent Cooperative Dynamic Scene Reconstruction T2 - 2025 IEEE Intelligent Vehicles Symposium (IV) UR - https://doi.org/10.1109/IV64158.2025.11097436 Y1 - 2025 UR - https://doi.org/10.1109/IV64158.2025.11097436 SN - 979-8-3315-3803-3 SP - 1033 EP - 1039 PB - IEEE CY - Piscataway ER -