TY - INPR A1 - Song, Rui A1 - Liu, Dai A1 - Chen, Dave Zhenyu A1 - Festag, Andreas A1 - Trinitis, Carsten A1 - Schulz, Martin A1 - Knoll, Alois T1 - Federated Learning via Decentralized Dataset Distillation in Resource Constrained Edge Environments N2 - In federated learning, all networked clients contribute to the model training cooperatively. However, with model sizes increasing, even sharing the trained partial models often leads to severe communication bottlenecks in underlying networks, especially when communicated iteratively. In this paper, we introduce a federated learning framework FedD3 requiring only one-shot communication by integrating dataset distillation instances. Instead of sharing model updates in other federated learning approaches, FedD3 allows the connected clients to distill the local datasets independently, and then aggregates those decentralized distilled datasets (e.g. a few unrecognizable images) from networks for model training. Our experimental results show that FedD3 significantly outperforms other federated learning frameworks in terms of needed communication volumes, while it provides the additional benefit to be able to balance the trade-off between accuracy and communication cost, depending on usage scenario or target dataset. For instance, for training an AlexNet model on CIFAR-10 with 10 clients under non-independent and identically distributed (Non-IID) setting, FedD3 can either increase the accuracy by over 71% with a similar communication volume, or save 98% of communication volume, while reaching the same accuracy, compared to other one-shot federated learning approaches. UR - https://doi.org/10.48550/arXiv.2208.11311 Y1 - 2022 UR - https://doi.org/10.48550/arXiv.2208.11311 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-60054 PB - arXiv CY - Ithaca ER - TY - INPR A1 - Song, Rui A1 - Liang, Chenwei A1 - Xia, Yan A1 - Zimmer, Walter A1 - Cao, Hu A1 - Caesar, Holger A1 - Festag, Andreas A1 - Knoll, Alois T1 - CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Driving N2 - Dynamic scene rendering opens new avenues in autonomous driving by enabling closed-loop simulations with photorealistic data, which is crucial for validating end-to-end algorithms. However, the complex and highly dynamic nature of traffic environments presents significant challenges in accurately rendering these scenes. In this paper, we introduce a novel 4D Gaussian Splatting (4DGS) approach, which incorporates context and temporal deformation awareness to improve dynamic scene rendering. Specifically, we employ a 2D semantic segmentation foundation model to self-supervise the 4D semantic features of Gaussians, ensuring meaningful contextual embedding. Simultaneously, we track the temporal deformation of each Gaussian across adjacent frames. By aggregating and encoding both semantic and temporal deformation features, each Gaussian is equipped with cues for potential deformation compensation within 3D space, facilitating a more precise representation of dynamic scenes. Experimental results show that our method improves 4DGS's ability to capture fine details in dynamic scene rendering for autonomous driving and outperforms other self-supervised methods in 4D reconstruction and novel view synthesis. Furthermore, CoDa-4DGS deforms semantic features with each Gaussian, enabling broader applications. UR - https://doi.org/10.48550/arXiv.2503.06744 Y1 - 2025 UR - https://doi.org/10.48550/arXiv.2503.06744 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-58313 PB - arXiv CY - Ithaca ER - TY - INPR A1 - Han, Longfei A1 - Xu, Qiuyu A1 - Kefferpütz, Klaus A1 - Elger, Gordon A1 - Beyerer, Jürgen T1 - Applying Extended Object Tracking for Self-Localization of Roadside Radar Sensors N2 - Intelligent Transportation Systems (ITS) can benefit from roadside 4D mmWave radar sensors for large-scale traffic monitoring due to their weatherproof functionality, long sensing range and low manufacturing cost. However, the localization method using external measurement devices has limitations in urban environments. Furthermore, if the sensor mount exhibits changes due to environmental influences, they cannot be corrected when the measurement is performed only during the installation. In this paper, we propose self-localization of roadside radar data using Extended Object Tracking (EOT). The method analyses both the tracked trajectories of the vehicles observed by the sensor and the aerial laser scan of city streets, assigns labels of driving behaviors such as "straight ahead", "left turn", "right turn" to trajectory sections and road segments, and performs Semantic Iterative Closest Points (SICP) algorithm to register the point cloud. The method exploits the result from a down stream task -- object tracking -- for localization. We demonstrate high accuracy in the sub-meter range along with very low orientation error. The method also shows good data efficiency. The evaluation is done in both simulation and real-world tests. UR - https://doi.org/10.48550/arXiv.2407.03084 Y1 - 2024 UR - https://doi.org/10.48550/arXiv.2407.03084 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-58361 PB - arXiv CY - Ithaca ER - TY - INPR A1 - Han, Longfei A1 - Kefferpütz, Klaus A1 - Beyerer, Jürgen T1 - 3D Extended Object Tracking based on Extruded B-Spline Side View Profiles N2 - Object tracking is an essential task for autonomous systems. With the advancement of 3D sensors, these systems can better perceive their surroundings using effective 3D Extended Object Tracking (EOT) methods. Based on the observation that common road users are symmetrical on the right and left sides in the traveling direction, we focus on the side view profile of the object. In order to leverage of the development in 2D EOT and balance the number of parameters of a shape model in the tracking algorithms, we propose a method for 3D extended object tracking (EOT) by describing the side view profile of the object with B-spline curves and forming an extrusion to obtain a 3D extent. The use of B-spline curves exploits their flexible representation power by allowing the control points to move freely. The algorithm is developed into an Extended Kalman Filter (EKF). For a through evaluation of this method, we use simulated traffic scenario of different vehicle models and realworld open dataset containing both radar and lidar data. UR - https://doi.org/10.48550/arXiv.2503.10730 Y1 - 2025 UR - https://doi.org/10.48550/arXiv.2503.10730 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-58331 PB - arXiv CY - Ithaca ER - TY - CHAP A1 - Agrawal, Shiva A1 - Bhanderi, Savankumar A1 - Amanagi, Sumit A1 - Doycheva, Kristina A1 - Elger, Gordon ED - Vinel, Alexey ED - Ploeg, Jeroen ED - Berns, Karsten ED - Gusikhin, Oleg T1 - Instance Segmentation and Detection of Children to Safeguard Vulnerable Traffic User by Infrastructure T2 - Proceedings of the 9th International Conference on Vehicle Technology and Intelligent Transport Systems N2 - Cameras mounted on intelligent roadside infrastructure units and vehicles can detect humans on the road using state-of-the-art perception algorithms, but these algorithms are presently not trained to distinguish between human and adult. However, this is a crucial requirement from a safety perspective because a child may not follow all the traffic rules, particularly while crossing the road. Moreover, a child may stop or may start playing on the road. In such situations, the separation of a child from an adult is necessary. The work in this paper targets to solve this problem by applying a transfer-learning-based neural network approach to classify child and adult separately in camera images. The described work is comprised of image data collection, data annotation, transfer learning-based model development, and evaluation. For the work, Mask-RCNN (region-based convolutional neural network) with different backbone architectures and two different baselines are investigated and the perception precision of the architectures after transfer-learning is compared. The results reveal that the best performing trained model is able to detect and classify children and adults separately in different road scenarios with segmentation mask AP (average precision) of 85% and bounding box AP of 92%. UR - https://doi.org/10.5220/0011825400003479 KW - Child and Adult Detection KW - Classification KW - Intelligent Roadside Infrastructure KW - Image Segmentation KW - Mask-RCNN KW - Traffic Flow Optimization KW - Transfer Learning Y1 - 2023 UR - https://doi.org/10.5220/0011825400003479 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-41785 SN - 978-989-758-652-1 SN - 2184-495X SP - 206 EP - 214 PB - SciTePress CY - Setúbal ER - TY - JOUR A1 - Agrawal, Shiva A1 - Bhanderi, Savankumar A1 - Elger, Gordon T1 - Infra-3DRC-FusionNet: Deep Fusion of Roadside Mounted RGB Mono Camera and Three-Dimensional Automotive Radar for Traffic User Detection JF - Sensors N2 - Mono RGB cameras and automotive radar sensors provide a complementary information set that makes them excellent candidates for sensor data fusion to obtain robust traffic user detection. This has been widely used in the vehicle domain and recently introduced in roadside-mounted smart infrastructure-based road user detection. However, the performance of the most commonly used late fusion methods often degrades when the camera fails to detect road users in adverse environmental conditions. The solution is to fuse the data using deep neural networks at the early stage of the fusion pipeline to use the complete data provided by both sensors. Research has been carried out in this area, but is limited to vehicle-based sensor setups. Hence, this work proposes a novel deep neural network to jointly fuse RGB mono-camera images and 3D automotive radar point cloud data to obtain enhanced traffic user detection for the roadside-mounted smart infrastructure setup. Projected radar points are first used to generate anchors in image regions with a high likelihood of road users, including areas not visible to the camera. These anchors guide the prediction of 2D bounding boxes, object categories, and confidence scores. Valid detections are then used to segment radar points by instance, and the results are post-processed to produce final road user detections in the ground plane. The trained model is evaluated for different light and weather conditions using ground truth data from a lidar sensor. It provides a precision of 92%, recall of 78%, and F1-score of 85%. The proposed deep fusion methodology has 33%, 6%, and 21% absolute improvement in precision, recall, and F1-score, respectively, compared to object-level spatial fusion output. UR - https://doi.org/10.3390/s25113422 Y1 - 2025 UR - https://doi.org/10.3390/s25113422 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-59421 SN - 1424-8220 VL - 25 IS - 11 PB - MDPI CY - Basel ER - TY - JOUR A1 - Da Rosa Zanatta, Mateus A1 - Carvalho Lustosa da Costa, Joao Paulo A1 - Antreich, Felix A1 - Haardt, Martin A1 - Elger, Gordon A1 - Lopes de Mendonça, Fábio Lúcio A1 - de Sousa Junior, Rafael Timóteo T1 - Tensor-based framework with model order selection and high accuracy factor decomposition for time-delay estimation in dynamic multipath scenarios JF - IEEE Access N2 - Global Navigation Satellite Systems (GNSS) are crucial for applications that demand very accurate positioning. Tensor-based time-delay estimation methods, such as CPD-GEVD, DoA/KRF, and SECSI, combined with the GPS3 L1C signal, are capable of, significantly, mitigating the positioning degradation caused by multipath components. However, even though these schemes require an estimated model order, they assume that the number of multipath components is constant. In GNSS applications, the number of multipath components is time-varying in dynamic scenarios. Thus, in this paper, we propose a tensor-based framework with model order selection and high accuracy factor decomposition for time-delay estimation in dynamic multipath scenarios. Our proposed approach exploits the estimates of the model order for each slice by grouping the data tensor slices into sub-tensors to provide high accuracy factor decomposition. We further enhance the proposed approach by incorporating the tensor-based Multiple Denoising (MuDe). UR - https://doi.org/10.1109/ACCESS.2020.3024597 KW - Global navigation satellite systems (GNSS) KW - global positioning system (GPS) KW - GPS3 KW - time-delay estimation (TDE) KW - multipath components KW - model order selection (MOS) Y1 - 2020 UR - https://doi.org/10.1109/ACCESS.2020.3024597 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-9528 SN - 2169-3536 VL - 8 SP - 174931 EP - 174942 PB - IEEE CY - New York ER - TY - JOUR A1 - Delooz, Quentin A1 - Willecke, Alexander A1 - Garlichs, Keno A1 - Hagau, Andreas-Christian A1 - Wolf, Lars A1 - Vinel, Alexey A1 - Festag, Andreas T1 - Analysis and Evaluation of Information Redundancy Mitigation for V2X Collective Perception JF - IEEE Access N2 - Sensor data sharing enables vehicles to exchange locally perceived sensor data among each other and with the roadside infrastructure to increase their environmental awareness. It is commonly regarded as a next-generation vehicular communication service beyond the exchange of highly aggregated messages in the first generation. The approach is being considered in the European standardization process, where it relies on the exchange of locally detected objects representing anything safety-relevant, such as other vehicles or pedestrians, in periodically broadcasted messages to vehicles in direct communication range. Objects filtering methods for inclusion in a message are necessary to avoid overloading a channel and provoking unnecessary data processing. Initial studies provided in a pre-standardization report about sensor data sharing elaborated a first set of rules to filter objects based on their characteristics, such as their dynamics or type. However, these rules still lack the consideration of information received by other stations to operate. Specifically, to address the problem of information redundancy, several rules have been proposed, but their performance has not been evaluated yet comprehensively. In the present work, the rules are further analyzed, assessed, and compared. Functional and operational requirements are investigated. A performance evaluation is realized by discrete-event simulations in a scenario for a representative city with realistic vehicle densities and mobility patterns. A score and other redundancy-level metrics are elaborated to ease the evaluation and comparison of the filtering rules. Finally, improvements and future works to the filtering methods are proposed. UR - https://doi.org/10.1109/ACCESS.2022.3170029 KW - Collective perception KW - information redundancy mitigation KW - road safety KW - sensor data sharing KW - V2X communications Y1 - 2022 UR - https://doi.org/10.1109/ACCESS.2022.3170029 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-28662 SN - 2169-3536 VL - 10 SP - 47076 EP - 47093 PB - IEEE CY - New York ER - TY - JOUR A1 - Song, Rui A1 - Zhou, Liguo A1 - Lyu, Lingjuan A1 - Festag, Andreas A1 - Knoll, Alois T1 - ResFed: Communication-Efficient Federated Learning With Deep Compressed Residuals JF - IEEE Internet of Things Journal N2 - Federated learning allows for cooperative training among distributed clients by sharing their locally learned model parameters, such as weights or gradients. However, as model size increases, the communication bandwidth required for deployment in wireless networks becomes a bottleneck. To address this, we propose a residual-based federated learning framework (ResFed) that transmits residuals instead of gradients or weights in networks. By predicting model updates at both clients and the server, residuals are calculated as the difference between updated and predicted models and contain more dense information than weights or gradients. We find that the residuals are less sensitive to an increasing compression ratio than other parameters, and hence use lossy compression techniques on residuals to improve communication efficiency for training in federated settings. With the same compression ratio, ResFed outperforms current methods (weight- or gradient-based federated learning) by over 1.4× on federated data sets, including MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100, and FEMNIST, in client-to-server communication, and can also be applied to reduce communication costs for server-to-client communication. UR - https://doi.org/10.1109/JIOT.2023.3324079 Y1 - 2023 UR - https://doi.org/10.1109/JIOT.2023.3324079 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46371 SN - 2327-4662 VL - 11 IS - 6 SP - 9458 EP - 9472 PB - IEEE CY - New York ER - TY - JOUR A1 - Hegde, Anupama A1 - Song, Rui A1 - Festag, Andreas T1 - Radio Resource Allocation in 5G-NR V2X: A Multi-Agent Actor-Critic Based Approach JF - IEEE Access N2 - The efficiency of radio resource allocation and scheduling procedures in Cellular Vehicle-to-X (Cellular V2X) communication networks directly affects link quality in terms of latency and reliability. However, owing to the continuous movement of vehicles, it is impossible to have a centralized coordinating unit at all times to manage the allocation of radio resources. In the unmanaged mode of the fifth generation new radio (5G-NR) V2X, the sensing-based semi-persistent scheduling (SB-SPS) loses its effectiveness when V2X data messages become aperiodic with varying data sizes. This leads to misinformed resource allocation decisions among vehicles and frequent resource collisions. To improve resource selection, this study formulates the Cellular V2X communication network as a decentralized multi-agent networked markov decision process (MDP) where each vehicle agent executes an actor-critic-based radio resource scheduler. Developing further the actor-critic methodology for the radio resource allocation problem in Cellular V2X, two variants are derived: independent actor-critic (IAC) and shared experience actor-critic (SEAC). Results from simulation studies indicate that the actor-critic schedulers improve reliability, achieving a 15-20% higher probability of reception under high vehicular density scenarios with aperiodic traffic patterns. UR - https://doi.org/10.1109/ACCESS.2023.3305267 KW - Cellular V2X KW - radio resource allocation KW - deep reinforcement learning Y1 - 2023 UR - https://doi.org/10.1109/ACCESS.2023.3305267 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-39352 SN - 2169-3536 VL - 11 SP - 87225 EP - 87244 PB - IEEE CY - New York ER - TY - JOUR A1 - Agrawal, Shiva A1 - Bhanderi, Savankumar A1 - Elger, Gordon T1 - Semi-Automatic Annotation of 3D Radar and Camera for Smart Infrastructure-Based Perception JF - IEEE Access N2 - Environment perception using camera, radar, and/or lidar sensors has significantly improved in the last few years because of deep learning-based methods. However, a large group of these methods fall into the category of supervised learning, which requires a considerable amount of annotated data. Due to uncertainties in multi-sensor data, automating the data labeling process is extremely challenging; hence, it is performed manually to a large extent. Even though full automation of such a process is difficult, semiautomation can be a significant step to ease this process. However, the available work in this regard is still very limited; hence, in this paper, a novel semi-automatic annotation methodology is developed for labeling RGB camera images and 3D automotive radar point cloud data using a smart infrastructure-based sensor setup. This paper also describes a new method for 3D radar background subtraction to remove clutter and a new object category, GROUP, for radar-based object detection for closely located vulnerable road users. To validate the work, a dataset named INFRA-3DRC is created using this methodology, where 75% of the labels are automatically generated. In addition, a radar cluster classifier and an image classifier are developed, trained, and tested on this dataset, achieving accuracy of 98.26% and 94.86%, respectively. The dataset and Python scripts are available at https://fraunhoferivi.github.io/INFRA-3DRC-Dataset/. UR - https://doi.org/10.1109/ACCESS.2024.3373310 Y1 - 2024 UR - https://doi.org/10.1109/ACCESS.2024.3373310 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46180 SN - 2169-3536 VL - 12 SP - 34325 EP - 34341 PB - IEEE CY - New York ER -