Open Access

Remarkable progress has recently occurred in the field of reinforcement learning (RL), especially with the development of groundbreaking deep reinforcement learning (DRL) solutions. DRL demonstrates outstanding potential in automating complex decision-making and control tasks, such as autonomous driving and robotics manipulation. However, numerous challenges persist, necessitating further research to effectively apply these methods in specific use cases. One of the major challenges in deploying DRL methods in safety-critical real-world settings is ensuring the system's reliability in handling various situations encountered during production use. This thesis focuses on investigating the behavior of a self-driving vehicle (SDV) trained using the Proximal Policy Optimization (PPO) algorithm and ensemble learning. We conducted the training of the SDV using DriverGym, an open-source OpenAI Gym-compatible environment designed for developing RL algorithms for autonomous driving with real-world scenarios. Our approach involved an ensemble-based DRL strategy using PPO. We aimed to investigate whether we could improve the model's performance and estimate epistemic uncertainty with the ensemble-based model. We explored the creation of an ensemble of multiple policies using a snapshot ensemble technique, without incurring additional training costs. Additionally, we also conducted a few experiments with an ensemble using the traditional random initialization approach. Compared to the performance of a single member, in terms of the average displacement error, the snapshot-based ensemble model generally exhibited improvements. Additionally, it provided a means to estimate epistemic uncertainty, allowing for the identification of out-of-distribution (OOD) driving scenes that the SDV had not encountered during training. Initial results from a traditional ensemble using random initialization suggest that the increased diversity this approach can offer may lead to a better estimate of epistemic uncertainty. One potential application of this ensemble-based uncertainty estimation approach is the identification of challenging driving scenarios through virtual simulators, which can be valuable for validating SDV planning and decision-making models. Another important application is to provide the trained system with enhanced confidence in handling previously unseen situations, especially in safety-critical contexts. We expect that our study and approach will contribute to advancing DRL models for autonomous systems, providing valuable insights and inspiring future research, particularly to enhance the capabilities and reliability of autonomous driving decision-making systems.

Data from production environments is now available in unprecedented volumes, making the problem-solving of incidents through root cause analysis straightforward. However, the root cause analysis process remains time-consuming. This study employs the Kitchenham standard systematic literature review methodology to explore how information models and deep learning can streamline this process. By conducting a comprehensive search across four major databases, we evaluate the current technological advancements and their application in root cause analysis. The aim of this study is to assesses the impact of information models for root cause analysis in a production environment. Our findings reveal that integrating knowledge graphs, association rule mining, and deep learning algorithms significantly improves the speed and depth of root cause analysis compared to traditional methods. Specifically, the use of neural networks in recent literature shows substantial advancements in analyzing complex datasets, facilitating large-scale data integration, and enabling automated learning capabilities. Comparing our findings with other recent studies highlights the advantages of using information modeling and deep learning technologies in root cause analysis. This comparison underscores the superior accuracy and efficiency of these advanced methodologies over traditional manual interpretation methods. The effective implementation of these technologies requires a robust foundation of clean, standardized data, giving rise to the concept of “Production IT.” Furthermore, it is crucial for this data to be openly available to facilitate academic research, thereby enabling the development of new methods for more efficient and effective root cause analysis.

Multidimensional Magnetic Resonance Imaging (MRI) is a versatile tool for microstructure mapping. We use a diffusion weighted inversion-recovery spin echo (DW-IR-SE) sequence with spiral readouts at ultra-strong gradients to acquire a rich diffusion-relaxation data set with sensitivity to myelin water. We reconstruct 1D and 2D spectra with a two-step convex optimization approach and investigate a variety of multidimensional MRI methods, including 1D multi-component relaxometry, 1D multi-component diffusometry, 2D relaxation correlation imaging, and 2D diffusion-relaxation correlation spectroscopic imaging (DR-CSI), in terms of their potential to quantify tissue microstructure, including the myelin water fraction (MWF). We observe a distinct spectral peak that we attribute to myelin water in multi-component T1 relaxometry, T1-T2 correlation, T1-D correlation, and T2-D correlation imaging. Due to lower achievable echo times compared to diffusometry, MWF maps from relaxometry have higher quality. While 1D multi-component T1 data allows much faster myelin mapping, 2D approaches could offer unique insights into tissue microstructure and especially myelin diffusion.

In this study, we introduce the ToolSurface-144 dataset, which is presented here for the first time. It comprises four subsets – Full R, Full S, Top R, and Top S – each containing 144 tool classes captured under varying illumination conditions and fields of view. A newly developed, patented imaging approach was employed to acquire the data. It is compared with conventional diffuse ring illumination to assess its effectiveness in evaluating state-of-the-art convolutional neural networks. This enabled a more targeted investigation of the role of global shape characteristics such as silhouettes versus localized features like the tool face, cutting edges, and delicate geometrical structures under different training strategies. In this study, we evaluate six state-of-the-art convolutional neural networks—AlexNet, DenseNet161, EfficientNet-B0, ResNet152, ResNet50, and VGG16—using three training strategies: fine-tuning, freezing of pre-trained layers, and training from scratch. The results show that EfficientNet-B0 consistently achieved the highest classification accuracy in nearly all experiments and data sets. Especially using the fine-tuning training strategy, the model achieved 99% accuracy in tool classification. ResNet50 benefited greatly from fine-tuning and freezing, achieving a significant increase in performance compared to training from scratch. In contrast, ResNet152, AlexNet, and VGG16 consistently showed poor classification performance, indicating difficulties regarding learning and generalisation. The results show that diffuse illumination and complete tool views provide the best classification conditions, while restricted image sections with homogeneous illumination negatively affect model performance. Among the evaluated training strategies, fine-tuning proved the most efficient training method for developing CNN models for tool classification.

In public transport, arrival times are typically communicated as point forecasts, aiming to present precise estimates. However, current prediction models are unable to provide such precise and reliable estimates due to unpredictable events. This results in arrival times on passenger information systems appearing inaccurate due to the lack of communicated uncertainty. We therefore investigated interval forecasts as an alternative in an online study, aiming to better communicate uncertainty in arrival times. Our findings indicate that interval forecasts improve the communication of uncertainty. Further, user satisfaction was driven primarily by waiting time, and this relationship was moderated by the forecast concept. Point forecasts were only well received when the bus arrived as predicted, otherwise users preferred the broader interval forecasts. Participants valued accuracy over precision when judging arrival times.

Software-driven systems are becoming pivotal, underpinning the majority of innovations in contemporary automobiles. This workshop is poised to explore a plethora of topics pertinent to automotive software development. Attendees will engage in discourse on the most apt methodologies, techniques, and tools essential for navigating the foremost challenges faced by researchers and practitioners in the field.

Accurate and comprehensive semantic segmentation of Bird's Eye View (BEV) is essential for ensuring safe and proactive navigation in autonomous driving. Although cooperative perception has exceeded the detection capabilities of single-agent systems, prevalent camera-based algorithms in cooperative perception neglect valuable information derived from historical observations. This limitation becomes critical during sensor failures or communication issues as cooperative perception reverts to single-agent perception, leading to degraded performance and incomplete BEV segmentation maps. This paper introduces TempCoBEV, a temporal module designed to incorporate historical cues into current observations, thereby improving the quality and reliability of BEV map segmentations. We propose an importance-guided attention architecture to effectively integrate temporal information that prioritizes relevant properties for BEV map segmentation. TempCoBEV is an independent temporal module that seamlessly integrates into state-of-the-art camera-based cooperative perception models. We demonstrate through extensive experiments on the OPV2V dataset that TempCoBEV performs better than non-temporal models in predicting current and future BEV map segmentations, particularly in scenarios involving communication failures. We show the efficacy of TempCoBEV and its capability to integrate historical cues into the current BEV map, improving predictions under optimal communication conditions by up to 2% and under communication failures by up to 19%. The code will be published on GitHub.

In Reinforcement Learning (RL), certain states demand special attention due to their significant influence on outcomes; these are identified as critical states. The concept of criticality is essential for the development of effective and robust policies and to improve overall trust in RL agents in real-world applications like autonomous driving. The current paper takes a deep dive into criticality and studies the evolution of criticality throughout training. The experiments are conducted on a new, simple yet intuitive continuous cliff maze environment and the Highway-env autonomous driving environment. Here, a novel finding is reported that criticality is not only learnt by the agent but can also be unlearned. We hypothesize that diversity in experiences is necessary for effective criticality quantification which is majorly driven by the chosen exploration strategy. This close relationship between exploration and criticality is studied utilizing two different strategies namely the ex ponential ε-decay and the adaptive ε-decay. The study supports the idea that effective exploration plays a crucial role in accurately identifying and understanding critical states.

Background: Due to their diversity, complexity, and above all importance, safety-critical and dependable systems must be developed with special diligence. Criticality increases as these systems likely contain artificial intelligence (AI) components known for their uncertainty. As software and reference architectures form the backbone of any successful system, including safety-critical dependable systems with learning-enabled components, choosing the suitable architecture that guarantees safety despite uncertainties is of great eminence. Aim: We aim to provide the missing overview of all existing architectures, their contribution to safety, and their level of maturity in AI-based safety-critical systems. Method: To achieve this aim, we report a systematic mapping study. From a set of 1,639 primary studies, we selected 38 relevant studies dealing with safety assurance through software architecture in AI-based safety-critical systems. The selected studies were then examined using various criteria to answer the research questions and identify gaps in this area of research. Results: Our findings showed which architectures have been proposed and to what extent they have been implemented. Furthermore, we identified gaps in different application areas of those systems and explained these gaps with various arguments. Conclusion: As the AI trend continues to grow, the system complexity will inevitably increase, too. To ensure the lasting safety of the systems, we provide an overview of the state of the art, intending to identify best practices and research gaps and direct future research more focused.