TY - INPR A1 - Song, Rui A1 - Liu, Dai A1 - Chen, Dave Zhenyu A1 - Festag, Andreas A1 - Trinitis, Carsten A1 - Schulz, Martin A1 - Knoll, Alois T1 - Federated Learning via Decentralized Dataset Distillation in Resource Constrained Edge Environments N2 - In federated learning, all networked clients contribute to the model training cooperatively. However, with model sizes increasing, even sharing the trained partial models often leads to severe communication bottlenecks in underlying networks, especially when communicated iteratively. In this paper, we introduce a federated learning framework FedD3 requiring only one-shot communication by integrating dataset distillation instances. Instead of sharing model updates in other federated learning approaches, FedD3 allows the connected clients to distill the local datasets independently, and then aggregates those decentralized distilled datasets (e.g. a few unrecognizable images) from networks for model training. Our experimental results show that FedD3 significantly outperforms other federated learning frameworks in terms of needed communication volumes, while it provides the additional benefit to be able to balance the trade-off between accuracy and communication cost, depending on usage scenario or target dataset. For instance, for training an AlexNet model on CIFAR-10 with 10 clients under non-independent and identically distributed (Non-IID) setting, FedD3 can either increase the accuracy by over 71% with a similar communication volume, or save 98% of communication volume, while reaching the same accuracy, compared to other one-shot federated learning approaches. UR - https://doi.org/10.48550/arXiv.2208.11311 Y1 - 2022 UR - https://doi.org/10.48550/arXiv.2208.11311 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-60054 PB - arXiv CY - Ithaca ER - TY - INPR A1 - Song, Rui A1 - Liang, Chenwei A1 - Xia, Yan A1 - Zimmer, Walter A1 - Cao, Hu A1 - Caesar, Holger A1 - Festag, Andreas A1 - Knoll, Alois T1 - CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Driving N2 - Dynamic scene rendering opens new avenues in autonomous driving by enabling closed-loop simulations with photorealistic data, which is crucial for validating end-to-end algorithms. However, the complex and highly dynamic nature of traffic environments presents significant challenges in accurately rendering these scenes. In this paper, we introduce a novel 4D Gaussian Splatting (4DGS) approach, which incorporates context and temporal deformation awareness to improve dynamic scene rendering. Specifically, we employ a 2D semantic segmentation foundation model to self-supervise the 4D semantic features of Gaussians, ensuring meaningful contextual embedding. Simultaneously, we track the temporal deformation of each Gaussian across adjacent frames. By aggregating and encoding both semantic and temporal deformation features, each Gaussian is equipped with cues for potential deformation compensation within 3D space, facilitating a more precise representation of dynamic scenes. Experimental results show that our method improves 4DGS's ability to capture fine details in dynamic scene rendering for autonomous driving and outperforms other self-supervised methods in 4D reconstruction and novel view synthesis. Furthermore, CoDa-4DGS deforms semantic features with each Gaussian, enabling broader applications. UR - https://doi.org/10.48550/arXiv.2503.06744 Y1 - 2025 UR - https://doi.org/10.48550/arXiv.2503.06744 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-58313 PB - arXiv CY - Ithaca ER - TY - INPR A1 - Han, Longfei A1 - Xu, Qiuyu A1 - Kefferpütz, Klaus A1 - Elger, Gordon A1 - Beyerer, Jürgen T1 - Applying Extended Object Tracking for Self-Localization of Roadside Radar Sensors N2 - Intelligent Transportation Systems (ITS) can benefit from roadside 4D mmWave radar sensors for large-scale traffic monitoring due to their weatherproof functionality, long sensing range and low manufacturing cost. However, the localization method using external measurement devices has limitations in urban environments. Furthermore, if the sensor mount exhibits changes due to environmental influences, they cannot be corrected when the measurement is performed only during the installation. In this paper, we propose self-localization of roadside radar data using Extended Object Tracking (EOT). The method analyses both the tracked trajectories of the vehicles observed by the sensor and the aerial laser scan of city streets, assigns labels of driving behaviors such as "straight ahead", "left turn", "right turn" to trajectory sections and road segments, and performs Semantic Iterative Closest Points (SICP) algorithm to register the point cloud. The method exploits the result from a down stream task -- object tracking -- for localization. We demonstrate high accuracy in the sub-meter range along with very low orientation error. The method also shows good data efficiency. The evaluation is done in both simulation and real-world tests. UR - https://doi.org/10.48550/arXiv.2407.03084 Y1 - 2024 UR - https://doi.org/10.48550/arXiv.2407.03084 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-58361 PB - arXiv CY - Ithaca ER - TY - INPR A1 - Han, Longfei A1 - Kefferpütz, Klaus A1 - Beyerer, Jürgen T1 - 3D Extended Object Tracking based on Extruded B-Spline Side View Profiles N2 - Object tracking is an essential task for autonomous systems. With the advancement of 3D sensors, these systems can better perceive their surroundings using effective 3D Extended Object Tracking (EOT) methods. Based on the observation that common road users are symmetrical on the right and left sides in the traveling direction, we focus on the side view profile of the object. In order to leverage of the development in 2D EOT and balance the number of parameters of a shape model in the tracking algorithms, we propose a method for 3D extended object tracking (EOT) by describing the side view profile of the object with B-spline curves and forming an extrusion to obtain a 3D extent. The use of B-spline curves exploits their flexible representation power by allowing the control points to move freely. The algorithm is developed into an Extended Kalman Filter (EKF). For a through evaluation of this method, we use simulated traffic scenario of different vehicle models and realworld open dataset containing both radar and lidar data. UR - https://doi.org/10.48550/arXiv.2503.10730 Y1 - 2025 UR - https://doi.org/10.48550/arXiv.2503.10730 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-58331 PB - arXiv CY - Ithaca ER - TY - CHAP A1 - Agrawal, Shiva A1 - Bhanderi, Savankumar A1 - Amanagi, Sumit A1 - Doycheva, Kristina A1 - Elger, Gordon ED - Vinel, Alexey ED - Ploeg, Jeroen ED - Berns, Karsten ED - Gusikhin, Oleg T1 - Instance Segmentation and Detection of Children to Safeguard Vulnerable Traffic User by Infrastructure T2 - Proceedings of the 9th International Conference on Vehicle Technology and Intelligent Transport Systems N2 - Cameras mounted on intelligent roadside infrastructure units and vehicles can detect humans on the road using state-of-the-art perception algorithms, but these algorithms are presently not trained to distinguish between human and adult. However, this is a crucial requirement from a safety perspective because a child may not follow all the traffic rules, particularly while crossing the road. Moreover, a child may stop or may start playing on the road. In such situations, the separation of a child from an adult is necessary. The work in this paper targets to solve this problem by applying a transfer-learning-based neural network approach to classify child and adult separately in camera images. The described work is comprised of image data collection, data annotation, transfer learning-based model development, and evaluation. For the work, Mask-RCNN (region-based convolutional neural network) with different backbone architectures and two different baselines are investigated and the perception precision of the architectures after transfer-learning is compared. The results reveal that the best performing trained model is able to detect and classify children and adults separately in different road scenarios with segmentation mask AP (average precision) of 85% and bounding box AP of 92%. UR - https://doi.org/10.5220/0011825400003479 KW - Child and Adult Detection KW - Classification KW - Intelligent Roadside Infrastructure KW - Image Segmentation KW - Mask-RCNN KW - Traffic Flow Optimization KW - Transfer Learning Y1 - 2023 UR - https://doi.org/10.5220/0011825400003479 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-41785 SN - 978-989-758-652-1 SN - 2184-495X SP - 206 EP - 214 PB - SciTePress CY - Setúbal ER - TY - JOUR A1 - Agrawal, Shiva A1 - Bhanderi, Savankumar A1 - Elger, Gordon T1 - Infra-3DRC-FusionNet: Deep Fusion of Roadside Mounted RGB Mono Camera and Three-Dimensional Automotive Radar for Traffic User Detection JF - Sensors N2 - Mono RGB cameras and automotive radar sensors provide a complementary information set that makes them excellent candidates for sensor data fusion to obtain robust traffic user detection. This has been widely used in the vehicle domain and recently introduced in roadside-mounted smart infrastructure-based road user detection. However, the performance of the most commonly used late fusion methods often degrades when the camera fails to detect road users in adverse environmental conditions. The solution is to fuse the data using deep neural networks at the early stage of the fusion pipeline to use the complete data provided by both sensors. Research has been carried out in this area, but is limited to vehicle-based sensor setups. Hence, this work proposes a novel deep neural network to jointly fuse RGB mono-camera images and 3D automotive radar point cloud data to obtain enhanced traffic user detection for the roadside-mounted smart infrastructure setup. Projected radar points are first used to generate anchors in image regions with a high likelihood of road users, including areas not visible to the camera. These anchors guide the prediction of 2D bounding boxes, object categories, and confidence scores. Valid detections are then used to segment radar points by instance, and the results are post-processed to produce final road user detections in the ground plane. The trained model is evaluated for different light and weather conditions using ground truth data from a lidar sensor. It provides a precision of 92%, recall of 78%, and F1-score of 85%. The proposed deep fusion methodology has 33%, 6%, and 21% absolute improvement in precision, recall, and F1-score, respectively, compared to object-level spatial fusion output. UR - https://doi.org/10.3390/s25113422 Y1 - 2025 UR - https://doi.org/10.3390/s25113422 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-59421 SN - 1424-8220 VL - 25 IS - 11 PB - MDPI CY - Basel ER - TY - JOUR A1 - Da Rosa Zanatta, Mateus A1 - Carvalho Lustosa da Costa, Joao Paulo A1 - Antreich, Felix A1 - Haardt, Martin A1 - Elger, Gordon A1 - Lopes de Mendonça, Fábio Lúcio A1 - de Sousa Junior, Rafael Timóteo T1 - Tensor-based framework with model order selection and high accuracy factor decomposition for time-delay estimation in dynamic multipath scenarios JF - IEEE Access N2 - Global Navigation Satellite Systems (GNSS) are crucial for applications that demand very accurate positioning. Tensor-based time-delay estimation methods, such as CPD-GEVD, DoA/KRF, and SECSI, combined with the GPS3 L1C signal, are capable of, significantly, mitigating the positioning degradation caused by multipath components. However, even though these schemes require an estimated model order, they assume that the number of multipath components is constant. In GNSS applications, the number of multipath components is time-varying in dynamic scenarios. Thus, in this paper, we propose a tensor-based framework with model order selection and high accuracy factor decomposition for time-delay estimation in dynamic multipath scenarios. Our proposed approach exploits the estimates of the model order for each slice by grouping the data tensor slices into sub-tensors to provide high accuracy factor decomposition. We further enhance the proposed approach by incorporating the tensor-based Multiple Denoising (MuDe). UR - https://doi.org/10.1109/ACCESS.2020.3024597 KW - Global navigation satellite systems (GNSS) KW - global positioning system (GPS) KW - GPS3 KW - time-delay estimation (TDE) KW - multipath components KW - model order selection (MOS) Y1 - 2020 UR - https://doi.org/10.1109/ACCESS.2020.3024597 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-9528 SN - 2169-3536 VL - 8 SP - 174931 EP - 174942 PB - IEEE CY - New York ER - TY - JOUR A1 - Delooz, Quentin A1 - Willecke, Alexander A1 - Garlichs, Keno A1 - Hagau, Andreas-Christian A1 - Wolf, Lars A1 - Vinel, Alexey A1 - Festag, Andreas T1 - Analysis and Evaluation of Information Redundancy Mitigation for V2X Collective Perception JF - IEEE Access N2 - Sensor data sharing enables vehicles to exchange locally perceived sensor data among each other and with the roadside infrastructure to increase their environmental awareness. It is commonly regarded as a next-generation vehicular communication service beyond the exchange of highly aggregated messages in the first generation. The approach is being considered in the European standardization process, where it relies on the exchange of locally detected objects representing anything safety-relevant, such as other vehicles or pedestrians, in periodically broadcasted messages to vehicles in direct communication range. Objects filtering methods for inclusion in a message are necessary to avoid overloading a channel and provoking unnecessary data processing. Initial studies provided in a pre-standardization report about sensor data sharing elaborated a first set of rules to filter objects based on their characteristics, such as their dynamics or type. However, these rules still lack the consideration of information received by other stations to operate. Specifically, to address the problem of information redundancy, several rules have been proposed, but their performance has not been evaluated yet comprehensively. In the present work, the rules are further analyzed, assessed, and compared. Functional and operational requirements are investigated. A performance evaluation is realized by discrete-event simulations in a scenario for a representative city with realistic vehicle densities and mobility patterns. A score and other redundancy-level metrics are elaborated to ease the evaluation and comparison of the filtering rules. Finally, improvements and future works to the filtering methods are proposed. UR - https://doi.org/10.1109/ACCESS.2022.3170029 KW - Collective perception KW - information redundancy mitigation KW - road safety KW - sensor data sharing KW - V2X communications Y1 - 2022 UR - https://doi.org/10.1109/ACCESS.2022.3170029 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-28662 SN - 2169-3536 VL - 10 SP - 47076 EP - 47093 PB - IEEE CY - New York ER - TY - JOUR A1 - Song, Rui A1 - Zhou, Liguo A1 - Lyu, Lingjuan A1 - Festag, Andreas A1 - Knoll, Alois T1 - ResFed: Communication-Efficient Federated Learning With Deep Compressed Residuals JF - IEEE Internet of Things Journal N2 - Federated learning allows for cooperative training among distributed clients by sharing their locally learned model parameters, such as weights or gradients. However, as model size increases, the communication bandwidth required for deployment in wireless networks becomes a bottleneck. To address this, we propose a residual-based federated learning framework (ResFed) that transmits residuals instead of gradients or weights in networks. By predicting model updates at both clients and the server, residuals are calculated as the difference between updated and predicted models and contain more dense information than weights or gradients. We find that the residuals are less sensitive to an increasing compression ratio than other parameters, and hence use lossy compression techniques on residuals to improve communication efficiency for training in federated settings. With the same compression ratio, ResFed outperforms current methods (weight- or gradient-based federated learning) by over 1.4× on federated data sets, including MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100, and FEMNIST, in client-to-server communication, and can also be applied to reduce communication costs for server-to-client communication. UR - https://doi.org/10.1109/JIOT.2023.3324079 Y1 - 2023 UR - https://doi.org/10.1109/JIOT.2023.3324079 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46371 SN - 2327-4662 VL - 11 IS - 6 SP - 9458 EP - 9472 PB - IEEE CY - New York ER - TY - JOUR A1 - Hegde, Anupama A1 - Song, Rui A1 - Festag, Andreas T1 - Radio Resource Allocation in 5G-NR V2X: A Multi-Agent Actor-Critic Based Approach JF - IEEE Access N2 - The efficiency of radio resource allocation and scheduling procedures in Cellular Vehicle-to-X (Cellular V2X) communication networks directly affects link quality in terms of latency and reliability. However, owing to the continuous movement of vehicles, it is impossible to have a centralized coordinating unit at all times to manage the allocation of radio resources. In the unmanaged mode of the fifth generation new radio (5G-NR) V2X, the sensing-based semi-persistent scheduling (SB-SPS) loses its effectiveness when V2X data messages become aperiodic with varying data sizes. This leads to misinformed resource allocation decisions among vehicles and frequent resource collisions. To improve resource selection, this study formulates the Cellular V2X communication network as a decentralized multi-agent networked markov decision process (MDP) where each vehicle agent executes an actor-critic-based radio resource scheduler. Developing further the actor-critic methodology for the radio resource allocation problem in Cellular V2X, two variants are derived: independent actor-critic (IAC) and shared experience actor-critic (SEAC). Results from simulation studies indicate that the actor-critic schedulers improve reliability, achieving a 15-20% higher probability of reception under high vehicular density scenarios with aperiodic traffic patterns. UR - https://doi.org/10.1109/ACCESS.2023.3305267 KW - Cellular V2X KW - radio resource allocation KW - deep reinforcement learning Y1 - 2023 UR - https://doi.org/10.1109/ACCESS.2023.3305267 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-39352 SN - 2169-3536 VL - 11 SP - 87225 EP - 87244 PB - IEEE CY - New York ER - TY - JOUR A1 - Agrawal, Shiva A1 - Bhanderi, Savankumar A1 - Elger, Gordon T1 - Semi-Automatic Annotation of 3D Radar and Camera for Smart Infrastructure-Based Perception JF - IEEE Access N2 - Environment perception using camera, radar, and/or lidar sensors has significantly improved in the last few years because of deep learning-based methods. However, a large group of these methods fall into the category of supervised learning, which requires a considerable amount of annotated data. Due to uncertainties in multi-sensor data, automating the data labeling process is extremely challenging; hence, it is performed manually to a large extent. Even though full automation of such a process is difficult, semiautomation can be a significant step to ease this process. However, the available work in this regard is still very limited; hence, in this paper, a novel semi-automatic annotation methodology is developed for labeling RGB camera images and 3D automotive radar point cloud data using a smart infrastructure-based sensor setup. This paper also describes a new method for 3D radar background subtraction to remove clutter and a new object category, GROUP, for radar-based object detection for closely located vulnerable road users. To validate the work, a dataset named INFRA-3DRC is created using this methodology, where 75% of the labels are automatically generated. In addition, a radar cluster classifier and an image classifier are developed, trained, and tested on this dataset, achieving accuracy of 98.26% and 94.86%, respectively. The dataset and Python scripts are available at https://fraunhoferivi.github.io/INFRA-3DRC-Dataset/. UR - https://doi.org/10.1109/ACCESS.2024.3373310 Y1 - 2024 UR - https://doi.org/10.1109/ACCESS.2024.3373310 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46180 SN - 2169-3536 VL - 12 SP - 34325 EP - 34341 PB - IEEE CY - New York ER - TY - CHAP A1 - Meess, Henri A1 - Gerner, Jeremias A1 - Hein, Daniel A1 - Schmidtner, Stefanie A1 - Elger, Gordon ED - Kieu, Minh Le ED - Dam, Koen H. van ED - Thompson, Jason ED - Malleson, Nick ED - Heppenstall, Alison ED - Ge, Jiaqi T1 - Real World Traffic Optimization by Reinforcement Learning: A Concept T2 - International Workshop on Agent-Based Modelling of Urban Systems (ABMUS) Proceedings: 2022 UR - https://doi.org/10.6084/m9.figshare.19733800.v1 Y1 - 2022 UR - https://doi.org/10.6084/m9.figshare.19733800.v1 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-29290 SP - 49 EP - 54 PB - figshare CY - [s. l.] ER - TY - JOUR A1 - Delooz, Quentin A1 - Festag, Andreas A1 - Vinel, Alexey T1 - Congestion Aware Objects Filtering for Collective Perception JF - Electronic Communications of the EASST N2 - This paper addresses collective perception for connected and automated driving. It proposes the adaptation of filtering rules based on the currently available channel resources, referred to as Enhanced DCC-Aware Filtering (EDAF). UR - http://dx.doi.org/10.14279/tuj.eceasst.80.1160 KW - V2X KW - Decentralized Congestion Control KW - Collective Perception Y1 - 2021 UR - http://dx.doi.org/10.14279/tuj.eceasst.80.1160 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-14761 SN - 1863-2122 VL - 80 PB - TU Berlin CY - Berlin ER - TY - JOUR A1 - Senel, Numan A1 - Kefferpütz, Klaus A1 - Doycheva, Kristina A1 - Elger, Gordon T1 - Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking JF - Processes N2 - Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving. UR - https://doi.org/10.3390/pr11020501 KW - environmental perception KW - sensor fusion KW - autonomous vehicle KW - unscented Kalman filter KW - object tracking KW - roadside units Y1 - 2023 UR - https://doi.org/10.3390/pr11020501 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-31989 SN - 2227-9717 VL - 11 IS - 2 PB - MDPI CY - Basel ER - TY - INPR A1 - Bazzi, Alessandro A1 - Sepulcre, Miguel A1 - Delooz, Quentin A1 - Festag, Andreas A1 - Vogt, Jonas A1 - Wieker, Horst A1 - Berens, Friedbert A1 - Spaanderman, Paul T1 - Multi-Channel Operation for the Release 2 of ETSI Cooperative Intelligent Transport Systems N2 - Vehicles and road infrastructure are starting to be equipped with vehicle-to-everything (V2X) communication solutions to increase road safety and provide new services to drivers and passengers. In Europe, the deployment is based on a set of Release 1 standards developed by ETSI to support basic use cases for cooperative intelligent transport systems (C-ITS). For them, the capacity of a single 10 MHz channel in the ITS band at 5.9 GHz is considered sufficient. At the same time, the ITS stakeholders are working towards several advanced use cases, which imply a significant increment of data traffic and the need for multiple channels. To address this issue, ETSI has recently standardized a new multi-channel operation (MCO) concept for flexible, efficient, and future-proof use of multiple channels. This new concept is defined in a set of new specifications that represent the foundation for the future releases of C-ITS standards. The present paper provides a comprehensive review of the new set of specifications, describing the main entities extending the C-ITS architecture at the different layers of the protocol stack, In addition, the paper provides representative examples that describe how these MCO standards will be used in the future and discusses some of the main open issues arising. The review and analysis of this paper facilitate the understanding and motivation of the new set of Release 2 ETSI specifications for MCO and the identification of new research opportunities. UR - https://doi.org/10.48550/arXiv.2305.19863 Y1 - 2023 UR - https://doi.org/10.48550/arXiv.2305.19863 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-41498 PB - arXiv CY - Ithaca ER - TY - JOUR A1 - Kettelgerdes, Marcel A1 - Sarmiento, Nicolas A1 - Erdogan, Hüseyin A1 - Wunderle, Bernhard A1 - Elger, Gordon T1 - Precise Adverse Weather Characterization by Deep-Learning-Based Noise Processing in Automotive LiDAR Sensors JF - Remote Sensing N2 - With current advances in automated driving, optical sensors like cameras and LiDARs are playing an increasingly important role in modern driver assistance systems. However, these sensors face challenges from adverse weather effects like fog and precipitation, which significantly degrade the sensor performance due to scattering effects in its optical path. Consequently, major efforts are being made to understand, model, and mitigate these effects. In this work, the reverse research question is investigated, demonstrating that these measurement effects can be exploited to predict occurring weather conditions by using state-of-the-art deep learning mechanisms. In order to do so, a variety of models have been developed and trained on a recorded multiseason dataset and benchmarked with respect to performance, model size, and required computational resources, showing that especially modern vision transformers achieve remarkable results in distinguishing up to 15 precipitation classes with an accuracy of 84.41% and predicting the corresponding precipitation rate with a mean absolute error of less than 0.47 mm/h, solely based on measurement noise. Therefore, this research may contribute to a cost-effective solution for characterizing precipitation with a commercial Flash LiDAR sensor, which can be implemented as a lightweight vehicle software feature to issue advanced driver warnings, adapt driving dynamics, or serve as a data quality measure for adaptive data preprocessing and fusion. UR - https://doi.org/10.3390/rs16132407 Y1 - 2024 UR - https://doi.org/10.3390/rs16132407 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-49599 SN - 2072-4292 VL - 16 IS - 13 PB - MDPI CY - Basel ER - TY - INPR A1 - Song, Rui A1 - Liang, Chenwei A1 - Cao, Hu A1 - Yan, Zhiran A1 - Zimmer, Walter A1 - Gross, Markus A1 - Festag, Andreas A1 - Knoll, Alois T1 - Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles N2 - Collaborative perception in automated vehicles leverages the exchange of information between agents, aiming to elevate perception results. Previous camera-based collaborative 3D perception methods typically employ 3D bounding boxes or bird's eye views as representations of the environment. However, these approaches fall short in offering a comprehensive 3D environmental prediction. To bridge this gap, we introduce the first method for collaborative 3D semantic occupancy prediction. Particularly, it improves local 3D semantic occupancy predictions by hybrid fusion of (i) semantic and occupancy task features, and (ii) compressed orthogonal attention features shared between vehicles. Additionally, due to the lack of a collaborative perception dataset designed for semantic occupancy prediction, we augment a current collaborative perception dataset to include 3D collaborative semantic occupancy labels for a more robust evaluation. The experimental findings highlight that: (i) our collaborative semantic occupancy predictions excel above the results from single vehicles by over 30%, and (ii) models anchored on semantic occupancy outpace state-of-the-art collaborative 3D detection techniques in subsequent perception applications, showcasing enhanced accuracy and enriched semantic-awareness in road environments. UR - https://doi.org/10.48550/arXiv.2402.07635 Y1 - 2024 UR - https://doi.org/10.48550/arXiv.2402.07635 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-46035 PB - arXiv CY - Ithaca ER - TY - JOUR A1 - Song, Rui A1 - Xu, Runsheng A1 - Festag, Andreas A1 - Ma, Jiaqi A1 - Knoll, Alois T1 - FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems JF - IEEE Transactions on Intelligent Vehicles N2 - Bird's eye view (BEV) perception is becoming increasingly important in the field of autonomous driving. It uses multi-view camera data to learn a transformer model that directly projects the perception of the road environment onto the BEV perspective. However, training a transformer model often requires a large amount of data, and as camera data for road traffic are often private, they are typically not shared. Federated learning offers a solution that enables clients to collaborate and train models without exchanging data but model parameters. In this paper, we introduce FedBEVT, a federated transformer learning approach for BEV perception. In order to address two common data heterogeneity issues in FedBEVT: (i) diverse sensor poses, and (ii) varying sensor numbers in perception systems, we propose two approaches - Federated Learning with Camera-Attentive Personalization (FedCaP) and Adaptive Multi-Camera Masking (AMCM), respectively. To evaluate our method in real-world settings, we create a dataset consisting of four typical federated use cases. Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving. UR - https://doi.org/10.1109/TIV.2023.3310674 Y1 - 2023 UR - https://doi.org/10.1109/TIV.2023.3310674 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-42789 SN - 2379-8904 SN - 2379-8858 VL - 9 IS - 1 SP - 958 EP - 969 PB - IEEE CY - New York ER - TY - CHAP A1 - Agrawal, Shiva A1 - Song, Rui A1 - Kohli, Akhil A1 - Korb, Andreas A1 - Andre, Maximilian A1 - Holzinger, Erik A1 - Elger, Gordon ED - Ploeg, Jeroen ED - Helfert, Markus ED - Berns, Karsten ED - Gusikhin, Oleg T1 - Concept of Smart Infrastructure for Connected Vehicle Assist and Traffic Flow Optimization T2 - Proceedings of the 8th International Conference on Vehicle Technology and Intelligent Transport Systems N2 - The smart infrastructure units can play a vital role to develop smart cities of the future and in assisting automated vehicles on the road by providing extended perception and timely warnings to avoid accidents. This paper focuses on the development of such an infrastructure unit, that is specifically designed for a pedestrian crossing junction. It can control traffic lights at the junction by real-time environment perception through its sensors and can optimize the flow of vehicles and passing vulnerable road users (VRUs). Moreover, it can assist on-road vehicles by providing real-time information and critical warnings via a v2x module. This paper further describes different use-cases of the work, all major hardware components involved in the development of smart infrastructure unit, referred to as an edge, different sensor fusion approaches using the camera, radar, and lidar mounted on the edge for environment perception, various modes of communication including v2x, system design UR - https://doi.org/10.5220/0011068800003191 KW - Road Side Unit (RSU) KW - Smart Infrastructure KW - Radar KW - LiDAR KW - Camera KW - Perception KW - Sensor Data Fusion KW - Vehicle-to-everything (V2X) KW - Communication KW - Simulation KW - ITS Y1 - 2022 UR - https://doi.org/10.5220/0011068800003191 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-30926 SN - 978-989-758-573-9 SN - 2184-495X SP - 360 EP - 367 PB - SciTePress CY - Setúbal ER - TY - CHAP A1 - Streck, Egor A1 - Herschel, Reinhold A1 - Wallrath, Patrick A1 - Sunderam, M. A1 - Elger, Gordon ED - Prasad, Venkatesha ED - Pesch, Dirk ED - Ansari, Nirwan ED - Benavente-Peces, César T1 - Comparison of Two Different Radar Concepts for Pedestrian Protection on Bus Stops T2 - Proceedings of the 11th International Conference on Sensor Networks N2 - This paper presents the joint work from the “HORIS” project, with a focus on pedestrian detection at bus-stops by radar sensors mounted in the infrastructure to support future autonomous driving and protecting pedestrians in critical situations. Two sensor systems are investigated and evaluated. The first based on single radar sensor phase-sensitive raw data analysis and the second based on sensor data fusion of cluster data with two radar sensors using neural networks to predict the position of pedestrians. UR - https://doi.org/10.5220/0010777100003118 KW - Sensor Data Fusion KW - Radar Sensor KW - Multiple-Sensor Systems KW - Machine Learning Y1 - 2022 UR - https://doi.org/10.5220/0010777100003118 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-30953 SN - 978-989-758-551-7 SN - 2184-4380 SP - 89 EP - 96 PB - SciTePress CY - Setúbal ER - TY - INPR A1 - Song, Rui A1 - Lyu, Lingjuan A1 - Jiang, Wei A1 - Festag, Andreas A1 - Knoll, Alois T1 - V2X-Boosted Federated Learning for Cooperative Intelligent Transportation Systems with Contextual Client Selection N2 - Machine learning (ML) has revolutionized transportation systems, enabling autonomous driving and smart traffic services. Federated learning (FL) overcomes privacy constraints by training ML models in distributed systems, exchanging model parameters instead of raw data. However, the dynamic states of connected vehicles affect the network connection quality and influence the FL performance. To tackle this challenge, we propose a contextual client selection pipeline that uses Vehicle-to-Everything (V2X) messages to select clients based on the predicted communication latency. The pipeline includes: (i) fusing V2X messages, (ii) predicting future traffic topology, (iii) pre-clustering clients based on local data distribution similarity, and (iv) selecting clients with minimal latency for future model aggregation. Experiments show that our pipeline outperforms baselines on various datasets, particularly in non-iid settings. UR - https://doi.org/10.48550/arXiv.2305.11654 Y1 - 2023 UR - https://doi.org/10.48550/arXiv.2305.11654 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-41441 PB - arXiv CY - Ithaca ER - TY - JOUR A1 - Bhanderi, Savankumar A1 - Agrawal, Shiva A1 - Elger, Gordon T1 - Deep segmentation of 3+1D radar point cloud for real-time roadside traffic user detection JF - Scientific Reports N2 - Smart cities rely on intelligent infrastructure to enhance road safety, optimize traffic flow, and enable vehicle-to-infrastructure (V2I) communication. A key component of such infrastructure is an efficient and real-time perception system that accurately detects diverse traffic participants. Among various sensing modalities, automotive radar is one of the best choices due to its robust performance in adverse weather and low-light conditions. However, due to low spatial resolution, traditional clustering-based approaches for radar object detection often struggle with vulnerable road user detection and nearby object separation. Hence, this paper proposes a deep learning-based D radar point cloud clustering methodology tailored for smart infrastructure-based perception applications. This approach first performs semantic segmentation of the radar point cloud, followed by instance segmentation to generate well-formed clusters with class labels using a deep neural network. It also detects single-point objects that conventional methods often miss. The described approach is developed and experimented using a smart infrastructure-based sensor setup and it performs segmentation of the point cloud in real-time. Experimental results demonstrate 95.35% F1-macro score for semantic segmentation and 91.03% mean average precision (mAP) at an intersection over union (IoU) threshold of 0.5 for instance segmentation. Further, the complete pipeline operates at 43.61 frames per second with a memory requirement of less than 0.7 MB on the edge device (Nvidia Jetson AGX Orin). UR - https://doi.org/10.1038/s41598-025-23019-6 Y1 - 2025 UR - https://doi.org/10.1038/s41598-025-23019-6 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-63458 SN - 2045-2322 VL - 15 PB - Springer Nature CY - London ER - TY - CHAP A1 - Maksimovski, Daniel A1 - Festag, Andreas A1 - Facchi, Christian T1 - A Survey on Decentralized Cooperative Maneuver Coordination for Connected and Automated Vehicles T2 - Proceedings of the 7th International Conference on Vehicle Technology and Intelligent Transport Systems N2 - V2X communications can be applied for maneuver coordination of automated vehicles, where the vehicles exchange messages to inform each other of their driving intentions and to negotiate for joint maneuvers. For motion and maneuver planning of automated vehicles, the cooperative maneuver coordination extends the perception range of the sensors, enhances the planning horizon and allows complex interactions among the vehicles. For specific scenarios, various schemes for maneuver coordination of connected automated vehicles exist. Recently, several proposals for maneuver coordination have been made that address generic instead of specific scenarios and apply different schemes for the message exchange of driving intentions and maneuver negotiation. This paper presents use cases for maneuver coordination and classifies existing generic approaches for decentralized maneuver coordination considering implicit and explicit trajectory broadcast, cost values and space-time reservation. We systematically describe the approaches, compare them and derive future research topics. UR - https://doi.org/10.5220/0010442501000111 KW - V2X Communications KW - Cooperative Driving KW - Maneuver Coordination KW - Automated Vehicle Y1 - 2021 UR - https://doi.org/10.5220/0010442501000111 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-14806 SN - 978-989-758-513-5 SN - 2184-495X SP - 100 EP - 111 PB - SciTePress CY - Setúbal ER - TY - CHAP A1 - Hegde, Anupama A1 - Stahl, Ringo A1 - Lobo, Silas A1 - Festag, Andreas T1 - Modeling Cellular Network Infrastructure in SUMO T2 - SUMO Conference Proceedings N2 - Communication networks are becoming an increasingly important part of the mobility system. They allow traffic participants to be connected and to exchange information related to traffic and roads. The information exchange impacts the behavior of traffic participants, such as the selection of travel routes or their mobility dynamics. Considering infrastructure-based networks, the information exchange depends on the availability of the network infrastructure and the quality of the communication links. Specifically in urban areas, today’s 4G and 5G networks deploy small cells of high capacity, which do not provide ubiquitous cellular coverage due to their small range, signal blocking, etc. Therefore, the accurate modeling of the network infrastructure and its integration in simulation scenarios in microscopic traffic simulation software is gaining relevance. Unlike traffic infrastructure, such as traffic lights, the simulation of a cellular network infrastructure is not natively supported in SUMO. Instead, the protocols, functions and entities of the communication system with the physical wireless transmission are modeled in a dedicated and specialized network simulator that is coupled with SUMO. The disadvantage of this approach is that the simulated SUMO entities, typically vehicles, are not aware which portions of the roads are covered by wireless cells and what quality the wireless communication links have. In this paper, we propose a method for modeling the cellular infrastructure in SUMO that introduces a cellular coverage layer to SUMO. This layer models cell sites in a regular hexagonal grid, where each site is served by a base station. Following commonly accepted guidelines for the evaluation of cellular communication system, the method facilitates standardized and realistic modeling of the cellular coverage, including cell sites, antenna characteristics, cell association and handover. In order to ease the applicability of the method, we describe the work flow to create cell sites. As a representative case, we have applied the method to InTAS, the SUMO Ingolstadt traffic scenario and applied real data for the cellular infrastructure. We validate the approach by simulating a Cellular V2X system with sidelink connectivity in an urban macro cell environment by coupling SUMO enhanced by the proposed connectivity sublayer with ARTERY-C, a network simulator for Cellular V2X. As a proof-of-concept, we present a signal-to-interference noise ratio (SINR) coverage map and further evaluate the impact of different types of interference. We also demonstrate the effect of advanced features of cellular networks such as inter-cell interference coordination (ICIC) and sidelink communication modes of Cellular V2X with dynamic switching between the in-coverage and out-of-coverage mode. UR - https://doi.org/10.52825/scp.v2i.97 Y1 - 2022 UR - https://doi.org/10.52825/scp.v2i.97 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-14785 SN - 2750-4425 VL - 2 SP - 99 EP - 113 PB - TIB Open Publishing CY - Hannover ER - TY - JOUR A1 - Schmid, Maximilian A1 - Zippelius, Andreas A1 - Hanß, Alexander A1 - Böckhorst, Stephan A1 - Elger, Gordon T1 - Investigations on High-Power LEDs and Solder Interconnects in Automotive Application: Part II - Reliability JF - IEEE Transactions on Device and Materials Reliability UR - https://doi.org/10.1109/TDMR.2023.3300355 KW - LED KW - non-destructive testing KW - reliability KW - solder KW - scanning acoustic microscopy (SAM) KW - thermal impedance (Zth) KW - thermal resistant (Rth) KW - X-ray KW - transient thermal analysis (TTA) Y1 - 2023 UR - https://doi.org/10.1109/TDMR.2023.3300355 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-39651 SN - 1558-2574 SN - 1530-4388 VL - 23 IS - 3 SP - 419 EP - 429 PB - IEEE CY - New York ER - TY - JOUR A1 - Meess, Henri A1 - Gerner, Jeremias A1 - Hein, Daniel A1 - Schmidtner, Stefanie A1 - Elger, Gordon A1 - Bogenberger, Klaus T1 - First steps towards real-world traffic signal control optimisation by reinforcement learning JF - Journal of Simulation N2 - Enhancing traffic signal optimisation has the potential to improve urban traffic flow without the need for expensive infrastructure modifications. While reinforcement learning (RL) techniques have demonstrated their effectiveness in simulations, their real-world implementation is still a challenge. Real-world systems need to be developed that guarantee a deployable action definition for real traffic systems while prioritising safety constraints and robust policies. This paper introduces a method to overcome this challenge by introducing a novel action definition that optimises parameter-level control programmes designed by traffic engineers. The complete proposed framework consists of a traffic situation estimation, a feature extractor, and a system that enables training on estimates of real-world traffic situations. Further multimodal optimisation, scalability, and continuous training after deployment could be achieved. The first simulative tests using this action definition show an average improvement of more than 20% in traffic flow compared to the baseline – the corresponding pre-optimised real-world control. UR - https://doi.org/10.1080/17477778.2024.2364715 Y1 - 2024 UR - https://doi.org/10.1080/17477778.2024.2364715 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-48585 SN - 1747-7778 VL - 18 IS - 6 SP - 957 EP - 972 PB - Taylor & Francis CY - London ER - TY - INPR A1 - Kettelgerdes, Marcel A1 - Hillmann, Tjorven A1 - Hirmer, Thomas A1 - Erdogan, Hüseyin A1 - Wunderle, Bernhard A1 - Elger, Gordon T1 - Accelerated Real-Life (ARL) Testing and Characterization of Automotive LiDAR Sensors to facilitate the Development and Validation of Enhanced Sensor Models N2 - In the realm of automated driving simulation and sensor modeling, the need for highly accurate sensor models is paramount for ensuring the reliability and safety of advanced driving assistance systems (ADAS). Hence, numerous works focus on the development of high-fidelity models of ADAS sensors, such as camera, Radar as well as modern LiDAR systems to simulate the sensor behavior in different driving scenarios, even under varying environmental conditions, considering for example adverse weather effects. However, aging effects of sensors, leading to suboptimal system performance, are mostly overlooked by current simulation techniques. This paper introduces a cutting-edge Hardware-in-the-Loop (HiL) test bench designed for the automated, accelerated aging and characterization of Automotive LiDAR sensors. The primary objective of this research is to address the aging effects of LiDAR sensors over the product life cycle, specifically focusing on aspects such as laser beam profile deterioration, output power reduction and intrinsic parameter drift, which are mostly neglected in current sensor models. By that, this proceeding research is intended to path the way, not only towards identifying and modeling respective degradation effects, but also to suggest quantitative model validation metrics. UR - https://doi.org/10.48550/arXiv.2312.04229 Y1 - 2023 UR - https://doi.org/10.48550/arXiv.2312.04229 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-59856 PB - arXiv CY - Ithaca ER - TY - JOUR A1 - Pandey, Amit A1 - Kühn, Stephan A1 - Weis, Alexander A1 - Wunderle, Bernhard A1 - Elger, Gordon T1 - Evaluating optical performance degradation of automotive cameras under accelerated aging JF - Optics and Lasers in Engineering N2 - Automotive cameras are subject to environmental stress, which degrades performance by reducing image sharpness. To qualify for automotive use and to ensure that the cameras maintain sharpness according to the hard requirements of end-of-line testing, cameras have to undergo standardized accelerated aging tests. These tests are performed to demonstrate reliability and functional safety over lifetime. Few studies have been published that demonstrate how aging contributes to the degradation of optical performance. This study addresses this gap by combining accelerated thermal aging with sharpness tracking to investigate degradation over time. To quantify sharpness degradation, six series-production cameras were subjected to accelerated thermal aging between −40◦𝐶 and +85◦𝐶. Each camera underwent 2000 aging cycles, equivalent to 80% of their lifetime based on the Coffin-Manson model of the LV124 standard. Sharpness was measured by calculating the Spatial Frequency Response (SFR) from images captured of a double-cross reticle projected by a virtual object generator with three illumination wavelengths (625nm, 520nm, and 470nm). The change in sharpness was evaluated with SFR50 and SFR at 60 line pairs per millimeter (SFR@60). During the first 250 cycles, a wear-in effect was observed, where sharpness increased before leveling off, as seen previously. The results also indicated a slow decline in sharpness showing long-term stability. Analysis indicated that before aging, the best focal plane was located closer to the focal position of the red wavelength, which lies furthest from the objective. By the end of the aging process, the best focal plane had shifted toward the focal position of the blue wavelength, which is located closer to the objective. This suggests a forward movement of the image sensor due to aging. Even after 2000 cycles, all cameras maintained an SFR@60 above 0.5. A Random Forest regression model was trained to predict the age based on the SFR curves, achieving a mean absolute error of 126 cycles and a 𝑅2 score of 0.96. UR - https://doi.org/10.1016/j.optlaseng.2025.109396 Y1 - 2025 UR - https://doi.org/10.1016/j.optlaseng.2025.109396 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-64821 SN - 1873-0302 VL - 2026 IS - 196 PB - Elsevier CY - Amsterdam ER -