Fraunhofer-Anwendungszentrum "Vernetzte Mobilität und Infrastruktur"
Refine
Document Type
- Conference Paper (34)
- Article (13)
- Preprint (3)
Institute
- Fraunhofer-Anwendungszentrum "Vernetzte Mobilität und Infrastruktur" (50)
- Fakultät Elektro- und Informationstechnik (47)
- CARISSMA Institute of Electric, Connected and Secure Mobility (C-ECOS) (24)
- Institut für Innovative Mobilität (IIMo) (21)
- AImotion Bavaria (3)
- Fakultät Informatik (3)
- Fakultät Elektrotechnik und Informatik (2)
Review
- peer-review (41)
- ja (5)
- nein (4)
Version
- published (18)
Federated learning allows for cooperative training among distributed clients by sharing their locally learned model parameters, such as weights or gradients. However, as model size increases, the communication bandwidth required for deployment in wireless networks becomes a bottleneck. To address this, we propose a residual-based federated learning framework (ResFed) that transmits residuals instead of gradients or weights in networks. By predicting model updates at both clients and the server, residuals are calculated as the difference between updated and predicted models and contain more dense information than weights or gradients. We find that the residuals are less sensitive to an increasing compression ratio than other parameters, and hence use lossy compression techniques on residuals to improve communication efficiency for training in federated settings. With the same compression ratio, ResFed outperforms current methods (weight- or gradient-based federated learning) by over 1.4× on federated data sets, including MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100, and FEMNIST, in client-to-server communication, and can also be applied to reduce communication costs for server-to-client communication.
Environment perception using camera, radar, and/or lidar sensors has significantly improved in the last few years because of deep learning-based methods. However, a large group of these methods fall into the category of supervised learning, which requires a considerable amount of annotated data. Due to uncertainties in multi-sensor data, automating the data labeling process is extremely challenging; hence,
it is performed manually to a large extent. Even though full automation of such a process is difficult, semiautomation can be a significant step to ease this process. However, the available work in this regard is still very limited; hence, in this paper, a novel semi-automatic annotation methodology is developed for labeling RGB camera images and 3D automotive radar point cloud data using a smart infrastructure-based sensor setup. This paper also describes a new method for 3D radar background subtraction to remove clutter and a new object category, GROUP, for radar-based object detection for closely located vulnerable road users. To validate the work, a dataset named INFRA-3DRC is created using this methodology, where 75% of the
labels are automatically generated. In addition, a radar cluster classifier and an image classifier are developed, trained, and tested on this dataset, achieving accuracy of 98.26% and 94.86%, respectively. The dataset and Python scripts are available at https://fraunhoferivi.github.io/INFRA-3DRC-Dataset/.
Collaborative perception in automated vehicles leverages the exchange of information between agents, aiming to elevate perception results. Previous camera-based collaborative 3D perception methods typically employ 3D bounding boxes or bird's eye views as representations of the environment. However, these approaches fall short in offering a comprehensive 3D environmental prediction. To bridge this gap, we introduce the first method for collaborative 3D semantic occupancy prediction. Particularly, it improves local 3D semantic occupancy predictions by hybrid fusion of (i) semantic and occupancy task features, and (ii) compressed orthogonal attention features shared between vehicles. Additionally, due to the lack of a collaborative perception dataset designed for semantic occupancy prediction, we augment a current collaborative perception dataset to include 3D collaborative semantic occupancy labels for a more robust evaluation. The experimental findings highlight that: (i) our collaborative semantic occupancy predictions excel above the results from single vehicles by over 30%, and (ii) models anchored on semantic occupancy outpace state-of-the-art collaborative 3D detection techniques in subsequent perception applications, showcasing enhanced accuracy and enriched semantic-awareness in road environments.
Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.
Bird's eye view (BEV) perception is becoming increasingly important in the field of autonomous driving. It uses multi-view camera data to learn a transformer model that directly projects the perception of the road environment onto the BEV perspective. However, training a transformer model often requires a large amount of data, and as camera data for road traffic are often private, they are typically not shared. Federated learning offers a solution that enables clients to collaborate and train models without exchanging data but model parameters. In this paper, we introduce FedBEVT, a federated transformer learning approach for BEV perception. In order to address two common data heterogeneity issues in FedBEVT: (i) diverse sensor poses, and (ii) varying sensor numbers in perception systems, we propose two approaches - Federated Learning with Camera-Attentive Personalization (FedCaP) and Adaptive Multi-Camera Masking (AMCM), respectively. To evaluate our method in real-world settings, we create a dataset consisting of four typical federated use cases. Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving.
Cameras mounted on intelligent roadside infrastructure units and vehicles can detect humans on the road using state-of-the-art perception algorithms, but these algorithms are presently not trained to distinguish between human and adult. However, this is a crucial requirement from a safety perspective because a child may not follow all the traffic rules, particularly while crossing the road. Moreover, a child may stop or may start playing on the road. In such situations, the separation of a child from an adult is necessary. The work in this paper targets to solve this problem by applying a transfer-learning-based neural network approach to classify child and adult separately in camera images. The described work is comprised of image data collection, data annotation, transfer learning-based model development, and evaluation. For the work, Mask-RCNN (region-based convolutional neural network) with different backbone architectures and two different baselines are investigated and the perception precision of the architectures after transfer-learning is compared. The results reveal that the best performing trained model is able to detect and classify children and adults separately in different road scenarios with segmentation mask AP (average precision) of 85% and bounding box AP of 92%.
Vehicles and road infrastructure are starting to be equipped with vehicle-to-everything (V2X) communication solutions to increase road safety and provide new services to drivers and passengers. In Europe, the deployment is based on a set of Release 1 standards developed by ETSI to support basic use cases for cooperative intelligent transport systems (C-ITS). For them, the capacity of a single 10 MHz channel in the ITS band at 5.9 GHz is considered sufficient. At the same time, the ITS stakeholders are working towards several advanced use cases, which imply a significant increment of data traffic and the need for multiple channels. To address this issue, ETSI has recently standardized a new multi-channel operation (MCO) concept for flexible, efficient, and future-proof use of multiple channels. This new concept is defined in a set of new specifications that represent the foundation for the future releases of C-ITS standards. The present paper provides a comprehensive review of the new set of specifications, describing the main entities extending the C-ITS architecture at the different layers of the protocol stack, In addition, the paper provides representative examples that describe how these MCO standards will be used in the future and discusses some of the main open issues arising. The review and analysis of this paper facilitate the understanding and motivation of the new set of Release 2 ETSI specifications for MCO and the identification of new research opportunities.
Machine learning (ML) has revolutionized transportation systems, enabling autonomous driving and smart traffic services. Federated learning (FL) overcomes privacy constraints by training ML models in distributed systems, exchanging model parameters instead of raw data. However, the dynamic states of connected vehicles affect the network connection quality and influence the FL performance. To tackle this challenge, we propose a contextual client selection pipeline that uses Vehicle-to-Everything (V2X) messages to select clients based on the predicted communication latency. The pipeline includes: (i) fusing V2X messages, (ii) predicting future traffic topology, (iii) pre-clustering clients based on local data distribution similarity, and (iv) selecting clients with minimal latency for future model aggregation. Experiments show that our pipeline outperforms baselines on various datasets, particularly in non-iid settings.
Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments
(2023)
The efficiency of radio resource allocation and scheduling procedures in Cellular Vehicle-to-X (Cellular V2X) communication networks directly affects link quality in terms of latency and reliability. However, owing to the continuous movement of vehicles, it is impossible to have a centralized coordinating unit at all times to manage the allocation of radio resources. In the unmanaged mode of the fifth generation new radio (5G-NR) V2X, the sensing-based semi-persistent scheduling (SB-SPS) loses its effectiveness when V2X data messages become aperiodic with varying data sizes. This leads to misinformed resource allocation decisions among vehicles and frequent resource collisions. To improve resource selection, this study formulates the Cellular V2X communication network as a decentralized multi-agent networked markov decision process (MDP) where each vehicle agent executes an actor-critic-based radio resource scheduler. Developing further the actor-critic methodology for the radio resource allocation problem in Cellular V2X, two variants are derived: independent actor-critic (IAC) and shared experience actor-critic (SEAC). Results from simulation studies indicate that the actor-critic schedulers improve reliability, achieving a 15-20% higher probability of reception under high vehicular density scenarios with aperiodic traffic patterns.
Radio Resource Allocation for Collective Perception in 5G-NR Vehicle-to-X Communication Systems
(2023)
Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.
This paper presents the joint work from the “HORIS” project, with a focus on pedestrian detection at bus-stops by radar sensors mounted in the infrastructure to support future autonomous driving and protecting pedestrians in critical situations. Two sensor systems are investigated and evaluated. The first based on single radar sensor phase-sensitive raw data analysis and the second based on sensor data fusion of cluster data with two radar sensors using neural networks to predict the position of pedestrians.
The smart infrastructure units can play a vital role to develop smart cities of the future and in assisting automated vehicles on the road by providing extended perception and timely warnings to avoid accidents. This paper focuses on the development of such an infrastructure unit, that is specifically designed for a pedestrian crossing junction. It can control traffic lights at the junction by real-time environment perception through its sensors and can optimize the flow of vehicles and passing vulnerable road users (VRUs). Moreover, it can assist on-road vehicles by providing real-time information and critical warnings via a v2x module. This paper further describes different use-cases of the work, all major hardware components involved in the development of smart infrastructure unit, referred to as an edge, different sensor fusion approaches using the camera, radar, and lidar mounted on the edge for environment perception, various modes of communication including v2x, system design
Sensor data sharing enables vehicles to exchange locally perceived sensor data among each other and with the roadside infrastructure to increase their environmental awareness. It is commonly regarded as a next-generation vehicular communication service beyond the exchange of highly aggregated messages in the first generation. The approach is being considered in the European standardization process, where it relies on the exchange of locally detected objects representing anything safety-relevant, such as other vehicles or pedestrians, in periodically broadcasted messages to vehicles in direct communication range. Objects filtering methods for inclusion in a message are necessary to avoid overloading a channel and provoking unnecessary data processing. Initial studies provided in a pre-standardization report about sensor data sharing elaborated a first set of rules to filter objects based on their characteristics, such as their dynamics or type. However, these rules still lack the consideration of information received by other stations to operate. Specifically, to address the problem of information redundancy, several rules have been proposed, but their performance has not been evaluated yet comprehensively. In the present work, the rules are further analyzed, assessed, and compared. Functional and operational requirements are investigated. A performance evaluation is realized by discrete-event simulations in a scenario for a representative city with realistic vehicle densities and mobility patterns. A score and other redundancy-level metrics are elaborated to ease the evaluation and comparison of the filtering rules. Finally, improvements and future works to the filtering methods are proposed.
Communication networks are becoming an increasingly important part of the mobility system. They allow traffic participants to be connected and to exchange information related to traffic and roads. The information exchange impacts the behavior of traffic participants, such as the selection of travel routes or their mobility dynamics. Considering infrastructure-based networks, the information exchange depends on the availability of the network infrastructure and the quality of the communication links. Specifically in urban areas, today’s 4G and 5G networks deploy small cells of high capacity, which do not provide ubiquitous cellular coverage due to their small range, signal blocking, etc. Therefore, the accurate modeling of the network infrastructure and its integration in simulation scenarios in microscopic traffic simulation software is gaining relevance.
Unlike traffic infrastructure, such as traffic lights, the simulation of a cellular network infrastructure is not natively supported in SUMO. Instead, the protocols, functions and entities of the communication system with the physical wireless transmission are modeled in a dedicated and specialized network simulator that is coupled with SUMO. The disadvantage of this approach is that the simulated SUMO entities, typically vehicles, are not aware which portions of the roads are covered by wireless cells and what quality the wireless communication links have.
In this paper, we propose a method for modeling the cellular infrastructure in SUMO that introduces a cellular coverage layer to SUMO. This layer models cell sites in a regular hexagonal grid, where each site is served by a base station. Following commonly accepted guidelines for the evaluation of cellular communication system, the method facilitates standardized and realistic modeling of the cellular coverage, including cell sites, antenna characteristics, cell association and handover. In order to ease the applicability of the method, we describe the work flow to create cell sites. As a representative case, we have applied the method to InTAS, the SUMO Ingolstadt traffic scenario and applied real data for the cellular infrastructure. We validate the approach by simulating a Cellular V2X system with sidelink connectivity in an urban macro cell environment by coupling SUMO enhanced by the proposed connectivity sublayer with ARTERY-C, a network simulator for Cellular V2X. As a proof-of-concept, we present a signal-to-interference noise ratio (SINR) coverage map and further evaluate the impact of different types of interference. We also demonstrate the effect of advanced features of cellular networks such as inter-cell interference coordination (ICIC) and sidelink communication modes of Cellular V2X with dynamic switching between the in-coverage and out-of-coverage mode.
Analysis of Existing Approaches for Information Sharing in Cooperative Intelligent Transport Systems
(2021)
A Survey on Decentralized Cooperative Maneuver Coordination for Connected and Automated Vehicles
(2021)
V2X communications can be applied for maneuver coordination of automated vehicles, where the vehicles exchange messages to inform each other of their driving intentions and to negotiate for joint maneuvers. For motion and maneuver planning of automated vehicles, the cooperative maneuver coordination extends the perception range of the sensors, enhances the planning horizon and allows complex interactions among the vehicles. For specific scenarios, various schemes for maneuver coordination of connected automated vehicles exist. Recently, several proposals for maneuver coordination have been made that address generic instead of specific scenarios and apply different schemes for the message exchange of driving intentions and maneuver negotiation. This paper presents use cases for maneuver coordination and classifies existing generic approaches for decentralized maneuver coordination considering implicit and explicit trajectory broadcast, cost values and space-time reservation. We systematically describe the approaches, compare them and derive future research topics.
Global Navigation Satellite Systems (GNSS) are crucial for applications that demand very accurate positioning. Tensor-based time-delay estimation methods, such as CPD-GEVD, DoA/KRF, and SECSI, combined with the GPS3 L1C signal, are capable of, significantly, mitigating the positioning degradation caused by multipath components. However, even though these schemes require an estimated model order, they assume that the number of multipath components is constant. In GNSS applications, the number of multipath components is time-varying in dynamic scenarios. Thus, in this paper, we propose a tensor-based framework with model order selection and high accuracy factor decomposition for time-delay estimation in dynamic multipath scenarios. Our proposed approach exploits the estimates of the model order for each slice by grouping the data tensor slices into sub-tensors to provide high accuracy factor decomposition. We further enhance the proposed approach by incorporating the tensor-based Multiple Denoising (MuDe).