Department Informatik
Refine
Year of publication
Document Type
- Doctoral Thesis (107)
- Report (19)
- Article (3)
- Moving Images (3)
- Conference Proceeding (2)
- Working Paper (2)
- Habilitation (1)
- Other (1)
Has Fulltext
- yes (138)
Keywords
- Maschinelles Lernen (7)
- Hochleistungsrechnen (6)
- Machine Learning (6)
- Simulation (6)
- Parallelisierung (5)
- Bildverarbeitung (4)
- Computertomographie (4)
- Medizinische Bildgebung (4)
- Mehrkernprozessor (4)
- - (3)
Even nowadays, people have only a very limited understanding about animal communication. Scientists are still far from identifying statistically relevant, animal-specific, and recurring linguistic paradigms. However, combined with the associated situation-specific behavioral observations, these patterns represent an indispensable basis for decoding animal communication. In order to derive statistically significant communicative and behavioral hypotheses, sufficiently large audiovisual data volumes are essential covering the animal-specific communicative and behavioral repertoire in a representative, natural, and realistic
manner. Hence, passive audiovisual monitoring techniques are increasingly deployed to obtain more natural insights, since the recording is performed in an unobtrusive fashion to minimize disruptive factors and simultaneously maximizing the probability of observing the entire inventory of natural communicative and behavioral paradigms in adequate numbers. Nevertheless, time- and human-resource constraints hamper scientists to efficiently process large-scale noise-heavy data archives, incorporating massive amounts of hidden audiovisual information, to derive an overall and bigger picture about animal linguistics. Thus, in
order to perform a deep and detailed data analysis to derive real-world representations, the support of machine-based data-driven algorithms is a fundamental prerequisite. In the scope of this doctoral thesis, a hybrid approach between machine (deep) learning and animal bioacoustics is presented, applying a wide variety of different and novel algorithms to analyze large-scale, noise-heavy, audiovisual, and animal-specific data repositories in order to provide completely new insights into the field of animal linguistics. Due to their complex social, communicative, and cognitive abilities, the largest member of the dolphin family – the killer whale (Orcinus orca) – was chosen as target species and prototype for this study. In northern British Columbia one of the largest animal-specific bioacoustic archives – the Orchive – was acquired by the OrcaLab and used as major data foundation, further extended by additional acoustic and behavioral data material, collected during project-internal fieldwork expeditions along the West Coast of Canada in 2017, 2018, 2019, and 2022. A broad spectrum of publicly available deep learning-based algorithms is presented, originally developed on killer whales, but also transferable to other vocalizing animal species, while addressing the following essential acoustic and image-related biological research questions: (1) signal segmentation – robust, efficient, and fully-automated detection of killer whale sound types,
(2) sound denoising – signal enhancement of diverse killer whale vocalizations, (3) call type identification – supervised, semi-supervised, and unsupervised deep architectures to recognize vocal killer whale paradigms, (4) sound type separation – signal segregation of overlapping killer whale vocalizations, (5) individual recognition – image-based deep learning framework to identify killer whale individuals, (6) sound source localization – underwater identification of vocalizing killer whale individuals, (7) signal generation – artificial and representative killer whale signal production, and (8) animal independence – adaption and generalization of developed killer whale-related deep learning concepts to other species-specific bioacoustic data volumes. All the inventive and publicly available machine (deep) learning frameworks
demonstrate auspicious results and provide totally unprecedented analysis techniques, facilitating more profound interpretations of massive, animal-specific, and audiovisual data volumes, all together building the imperative foundation to significantly push not only the communicative and behavioral understanding of killer whales, but the entire research field of animal bioacoustics and linguistics.
Executing system calls requires switching the privilege level when entering the kernel. This switch is considered essential to secure the operating system from unwanted access but is also costly. In other contexts, on-demand rather than proactive action allows paying overhead costs only when necessary. For example, non-blocking synchronization forgoes proactive locking in favor of retries if a conflict occurs.
We propose that single-address-space operating systems allow for an on-demand approach for system calls. When the operating-system data is hidden by the sheer size of the address space, elevated privileges are only necessary when performing privileged hardware operations. We believe that single-address-space systems would benefit from such an on-demand approach.
We introduce a concept which allows to defer the privilege-level switch until it is actually needed, that is when the first privileged instruction is encountered. We present a prototype implementation of the concept as an adaption of the Linux kernel as a widespread single-address-space kernel does not exist.
In a medical sense, radiography is an imaging technique to produce a plain image of the human body. Radiography of the chest is the most commonly used medical image acquisition method. In contrast to other imaging techniques, radiography benefits from fast examination procedures and low radiation doses. Moreover, a chest radiograph delivers detailed findings of diseases, pathologies, and abnormal structures in the thoracic and abdominal region. For example, fluid retention in the lung, an enlarged heart, or lung nodules specify only a fraction of the large variety of abnormalities visible in chest radiographs. With the breakout of the COVID-19 disease, an additional major use case was integrated into the clinical workflow for chest radiography assessment.
The high amount of produced radiographic images require a time-consuming reading process by clinical professionals. The daily workload and the connected liability often hinder the assurance of a consistent image reading accuracy. In other words, human-like behavior such as fatigue, concentration lack, and time pressure reduces performance.
To maintain a constant reading output, so-called computer-aided detection (CAD) systems are incorporated into the clinical workflow to compensate for the pitfalls of the human. Moreover, applying these systems not only simplifies the reading process for the experts but also reduces the reading time. In the beginning, the integrated systems were based on conventional and rule-based methods, e.g., image gradient filters. However, with the start of the deep learning (DL) era, including the availability of a huge number of training data, the performance of CAD systems rapidly increased. To leverage these systems, advanced systems are developed to increase the generalizability of such networks.
Especially lung nodules are challenging to read as it appears as a tiny fraction in the image. Due to the resulting low system performance, augmentation methods are designed to specifically focus on improving nodule recognition. For example, nodules of computed tomography (CT) images are leveraged to expand the nodule collection for training in the radiography domain.
The proposed approaches based on DL require a huge amount of training data. The demand for big chest radiography collections results in the ever-growing number of datasets publicly available. In practice, strict anonymization procedures are applied prior to the release of the data. For example, sensitive data such as patient names are removed. However, these anonymization methods may facilitate that the patient can be re-identified from the medical image content. For example, sensitive content can be linked between images by a system which identifies images from the same patient. This scenario highlights lacks in the current anonymization procedures as well as possible risks in data privacy and security.
Valvular heart disease (VHD) is a major public health problem with a prevalence of 2.5% in the US. Tens of thousands of minimally invasive procedures are performed every year and new therapeutical approaches are continuously developed at a rapid pace. Major clinical decisions and therapies are nowadays guided by peri- and intraoperative imaging with an increasing emphasis on 3D. Given the complexity of pathology and therapies in VHD, there is a strong need for fast, precise and reproducible quantification.
To this end, several contributions are proposed in this thesis across various aspects of VHD management. These are structured into two parts: i) Mitral valve (MV) modeling and ii) advanced computational decision support.
The first part starts with describing a comprehensive MV model obtained from 3D+t transesophageal echocardiography (TEE). An original methodology combines robust machine learning (ML) techniques and biomechanical constraints to obtain a temporally consistent estimate of model parameters. Extensive experiments on a large data set of 195 3D+t TEE volumes -- comprising 3026 3D frames -- demonstrate, that this automatic method is highly competitive in terms of speed and accuracy and outperforms purely data-driven methods in terms of robustness. Clinical evaluation in numerous centers around the world shows the applicability of its versatile quantification capabilities to various aspects of disease characterization and therapy planning.
Subsequently a more lightweight alternative enables for robust 3D tracking of the MV annulus at high frame rates. Two parallel components based on ML and image-based tracking complement each other in robustness and speed. Experiments with emulated probe motion suggest suitability towards an interventional setting for guiding transcatheter mitral valve repair (TMVR) procedures.
The second part proposes computational approaches for advanced decision support throughout the clinical workflow. First, the previously described valve modeling is combined with chamber models into a holistic and detailed model of the left heart. This enables for estimating patient-specific computational hemodynamics by serving as boundary condition for a level-set based computational fluid dynamics (CFD) solver. A validation concept using clinically acquired Doppler measurements is proposed and shows high agreement.
Finally, a framework is presented for post-operative modeling of self-expandable stent devices for monitoring in transcatheter aortic valve implantation (TAVI) procedures. The technique is based on deformable simplex meshes, geometrical constraints and ML. Evaluation on postoperative computed tomography (CT) data shows promising accuracy.
The continuous expansion of renewable energies poses a major challenge to the energy industry. In order to secure the power supply in the future, storage systems are inevitable. The widely used lithium-ion battery technology is efficient, but it is poorly scalable and also consists of toxic materials. An alternative is the so-called carnot battery, in which electricity is first converted into heat, stored in a thermal storage and converted back into electricity at a later time.
Since carnot batteries are a young technology with a low level of technological maturity, their application potential has been little explored. To better evaluate the potential, a simulation model was developed to study their application in different scenarios. The modular simulation model was validated with data from a pilot plant and a local energy supplier. The main focus of this work was to use the validated simulation model to investigate an application on house and grid level in more detail.
An analysis on house level showed that there is great potential for an application. It is possible to influence the operation of the carnot battery in such a way, that either only the electricity demand or additionally a heat demand at low temperature is covered. Especially in communities with various houses, this flexibility creates a savings potential of several hundred euros per house, depending on how the market conditions and investment costs develop in the future. On the other hand, an application on grid level, e.g. for an entire German county, is not recommended. The results showed that massive reductions in investment costs are necessary and that only systems, which are much larger than the currently planned prototypes, could generate profits at all with specific conditions.
Finally, a life cycle assessment was conducted. It showed that the carnot battery is significantly better for the environment than lithium-ion batteries, especially during production. Depending on the origin of the stored electricity (fossil or renewable), the carnot battery can keep this low environmental impact even during operation, despite a lower efficiency.
Automatic Generation of Test Scenarios and Simulation Scenarios based on Real-World Driving Data
(2023)
The functional safety of driver assistance systems as well as automated and connected driving functions must be ensured by the automotive manufacturers in every conceivable traffic situation. During the development and validation process, a considerable number of so-called driving scenarios must be tested. In the future, this extensive scope of testing can actually only be meaningfully managed by the massive use of simulation. In order to generate a corresponding validity and practical relevance in these simulations, models of the testing vehicle, the routes and environment as well as the surrounding traffic have to be modeled adequately. In the context of this work, real-world driving data will provide the key to achieving this level of capability. Test drives of real test vehicles are sensed and recorded. From these data recordings, the driving scenarios are firstly identified, then extracted and modeled in a driving simulation. In this way, the accuracy of the simulation model can be compared with the recorded data and validated. In addition, the collected driving scenarios can thus be performed and tested with an automated driving function in the simulation. This leads to a fully automated generation of test scenarios for the validation of automated driving functions.
Classical core memory was entirely non-
volatile and could keep at least part of the operating
system (OS) in main memory even across power cycles.
These days we can have terabytes of NVRAM to repeat
this approach, albeit on an entirely different scale and
with large parts of the OS state still kept in the volatile
CPU caches. In this paper, we discuss our experiences of
running large modern operating systems including their
applications entirely in NVRAM. We adapted stock Linux
and FreeBSD kernels to work exclusively with NVRAM by
hiding all DRAM from the kernels at boot time to establish
a realistic performance baseline without changing anything
else. Following this entirely NVRAM-agnostic approach,
we could observe an effective performance penalty of
a factor of about four, but only negligible increases in
whole-system power draw. For our system with two CPU
sockets and 56 cores total, we also observed a reduction
in power draw in several scenarios. Due to prolonged
execution times, the energy consumption increased as
well for these measured workloads. While this might be
discouraging at first sight, this result was achieved without
any performance tuning as to the specific characteristics
of today’s NVRAM technology. Therefore, we are also
discussing means to mitigate the observed shortcomings
by integrating NVRAM appropriately into the memory
hierarchy of future robust persistent systems.
In the pursuit of acquiring an immersive visual experience, camera technology has gone through a long evolution process. Modern digital cameras can capture high-resolution high dynamic range images with an extended depth of field. However, capturing just the 2D spatial information of the visual scene is not enough to deceive human perception for an immersive experience. Alternatively, light-field imaging technology allows for capturing the directional information of light rays together on top of the conventional 2D spatial data. This additional angular information in light-field images plays a pivotal role in many applications, including post-capture refocusing, depth estimation, 3D reconstruction, and novel view rendering. The thesis reviews the complete light-field processing pipeline which includes data capturing, depth estimation, and novel view synthesis. Moreover, inspired by the success of deep learning in different computer vision applications, this dissertation proposes to solve multiple issues concerning light-field processing with the help of neural networks. Starting with the data capturing, the first highresolution high dynamic range light-field dataset is captured for the community to develop and test their algorithms. Additionally, an initial study explains the effect of tone-mapping on view rendering quality. Quantitative and qualitative analysis indicates that tone-mapping after view rendering yields better results than applying it before rendering, especially in the presence of non-Lambertian objects. Moreover, disparity estimation is more reliable and accurate from raw HDR light-field than the tonemapped light-field.
Apart from HDR light fields, the thesis also presents a recurrent neural network predicting wrong disparity assignments due to the ill-posed nature of the problem. The proposed algorithm estimates a confidence value for each pixel location, filtering out the disparity outliers. The confidence for a given pixel is calculated only from its associated matching costs, without taking into account any additional nearby pixels, in order to keep a low complexity. These low-confidence pixels can then be corrected using reliable pixel values. Experimental results on multiple datasets show the robustness of the proposed method by outperforming state-of-the-art confidence estimation methods. Moreover, the size of the proposed confidence measure network in terms of the number of trainable parameters is almost 10^2-10^4 times less than state-of-the-art methods.
Despite filtering and fixing of incorrect disparity assignments, current light-field processing pipelines fail to reconstruct novel views with good quality. The thesis presents a learning-based lightfield view synthesis framework based on an end-to-end attention mechanism proposing a solution to the current pipeline shortcomings. The proposed framework consists of three convolutional neural networks connected sequentially, one network for stereo feature extraction, disparity estimation, and refinement each. The refinement network utilizes convolutional block attention modules in a residual network-style architecture to enhance depth image-based rendering. The initial design of the network renders a single virtual view, further extended by presenting two refinement network strategies to generate multiple light-field views. The proposed method performs better than state-of-the-art light-field rendering approaches, especially in occluded areas. Moreover, introducing the attention mechanism in the refinement stage helps preserving thin structures in the scene. The experimental results show that the proposed method generates consistent performance across diverse test datasets despite training on a content-specific dataset.
Novel view synthesis quality depends on the number of light-field perspective views used for the reconstruction. However, the redundant information in different light-field views poses challenges for storage and transmission resources. Motivated by the big advances in the field of image compression via machine learning, a compression of such data with the help of neural networks is desirable. However, neural network-based compression is a relatively new and immature research area lacking many basic functionalities, such as rate control. To close this gap, this thesis makes the first step toward multi-view compression by looking at stereo images with the purpose to reduce complexity and increase understanding. In particular, a novel recurrent neural network-based technique is proposed for stereo image compression with discrete rate control. The main contribution of the proposed method is to investigate how the redundancy between the images can be eliminated. A key technology is state
warping between the recurrent units of the stereo image networks to share mutual information. Additionally, a convolutional neural network utilizes compressed information to estimate occlusion maps, tackling discrepancies in the occluded areas. Quantitative and qualitative experimental analysis on two different datasets shows that the proposed technique saves 10-30% of the bit rate for the right image and outperforms conventional image codecs in terms of perceptual quality.
Over the last few decades, the demand for high-power optical systems continuously increased. High power is also accompanied by high temperature loads, which always induce undesired effects. This holds for the laser system creating the high-power beam and the optical system imaging the beam. This work presents a detailed simulation model for various temperature-induced optical effects. The model combines techniques for solving heat and structural mechanic equations and couples them with optical propagation methods, realized with ray tracing. It enables the predict the performance of thermally aberrated systems in physically realistic situations.
To achieve this, new ray tracing algorithms are presented, which allow the propagation through
anisotropic, inhomogeneous, and deformed optical media. The optical surface deformation is treated as a small perturbation of the initial, analytic surface, which leads to very accurate results for the deformation. Furthermore, methods for evaluating the effect of stress-induced weak anisotropy are presented, which can be used to estimate the magnitude of such influences. This is used to investigate whether bifocusing does affect general imaging systems and how large its contribution is in the worst case. It is also shown that the spatial walk-off does not significantly influence thermal lensing. In order to obtain accurate results concerning the field polarization, a Jones-Split method is presented, coupling ray tracing in an inhomogeneous and anisotropic media with the Jones formalism. Furthermore, an efficient model for the optical heat source is presented, which requires significantly less computational effort than other common methods. Also, a new method of interpolating continuous data on the unstructured grid is presented. The data interpolation for solving the presented ray equations poses a significant computational effort, which is why an efficient interpolation algorithm is crucial. The newly proposed normal projection method allows fast and accurate data interpolation without needing further fitting algorithms or auxiliary grids. The combined use of the proposed methods enables extensive parameter studies, allowing for the investigation of new design opportunities and compensation methods. Several experimental comparisons were conducted to validate the simulation model. This is done for a Nd:YAG amplifier, in which the specific state of polarization is measured in order to obtain data on depolarization and bifocusing. The thermal induced focal shift is also measured for a single lens and a complex optical scanner system. Excellent agreement between the simulation and the measured data could also be achieved in all cases.
As a result of this simulation method, new optical compensation setups which significantly reduce the depolarization losses in high-power solid-state amplifiers could be found. It is demonstrated that with [100]-cut Nd:YAG crystals, the depolarization can be reduced by more than a magnitude compared to [111]-cut crystals while achieving an overall better beam quality at the same time. Furthermore, new compensation schemes for optical imaging systems are proposed, which rely on a combination of optimized surface absorption and negative thermo-optic coefficients. These allow compensating for thermal lensing in general optical systems precisely. Also, it is shown that the stress-induced thermal lens significantly affects the optical properties and must not be neglected.
Criminal investigations often involve images that can provide important clues or serve as evidence in court. To validate the authenticity of an image and identify its source, a broad range of image forensic tools has been developed. For recent years, the most powerful of these tools have been based on machine learning. However, concerns about the reliability, security, and opacity of machine learning raise the question of whether such tools can be used in criminal investigations.
This thesis explores the practical applicability of machine learning in image forensics mainly from a technical but also from legal perspective. From a technical perspective, a major challenge behind machine learning tools is their sensitivity to training-test mismatches. To mitigate this issue, we propose the use of Bayesian detectors. While traditional detectors faced with unfamiliar inputs tend to fail silently, Bayesian detectors communicate the uncertainty in their prediction to the forensic analyst. Based on this predictive uncertainty, the forensic analyst can quantify how much to trust in the prediction.
We demonstrate the benefits of Bayesian detectors using three popular forensic tasks. First, we study Bayesian linear regression for estimating the scale factor from rescaled images. Bayesian linear regression achieves comparable performance to other methods, and the predictive uncertainty additionally enables detecting images with unknown scale factors and unseen post-processing.
Second, we study Bayesian logistic regression for the task of detecting JPEG double compression. The detector achieves high classification accuracy for known JPEG settings. Simultaneously, the predictive uncertainty exposes several pathological failure cases including training-test mismatches in quantization tables and in the JPEG encoder implementation.
Third, we study Gaussian processes for the task of camera model identification. Many previous methods assume that an image under analysis originates from a given set of known camera models, but in practice a photo can also come from an unknown camera model. To avoid misclassifications, Gaussian processes provide a rejection mechanism through their probabilistic predictions. We demonstrate that a Gaussian process classifier achieves high classification accuracy for known cameras and provides reliable uncertainty estimates for unknown cameras.
Another issue from a technical perspective is the vulnerability of machine learning to adversarial attacks. To harden forensic detectors against evasion attacks, previous work proposed to combine multiple classifiers to the one-and-a-half-class~(1.5C) classifier. To study its security in the white-box scenario, we propose a novel attack that requires only little image distortion. Our security analysis reveals three subtle pitfalls that undermine the security of the 1.5C classifier. We demonstrate that replacing the final component of the 1.5C classifier overcomes these pitfalls and achieves greater robustness.
From a legal perspective, the recently proposed Artificial Intelligence Act classifies the use of machine learning in law enforcement as high risk. Machine learning tools for high-risk applications are permitted but must meet mandatory requirements. We summarize these requirements and discuss their alignment with recent research on the two forensic applications of license plate recognition and deep fake detection. Our discussion highlights the key challenges and directions for future work towards legal compliance with the Artificial Intelligence Act.
Design and Implementation of an Automated Performance Modeling Toolkit for Regular Loop Kernels
(2023)
Analytical performance modeling helps software developers understand performance behavior and, as a result, guides development towards more efficient and faster software. Resulting models are the distilled knowledge and experience from performance and architecture analysis, poured into a structured form. Their construction and application is a complex, tedious and labor-intensive task, that requires in-depth knowledge of modern architectures and comes with potential pitfalls. This research aims to lower the entry barrier of using performance models to allow large-scale performance model studies and make analytic models accessible to code generation and similar use-cases. The focus lies on–but is not strictly limited to–regular loop kernels, which are defined by long-running loop nests with data-content independent performance behavior.
Techniques and implementations for static prediction of performance model contributions have been developed. They specifically cover the throughput of instruction execution and transfer volumes in cache architectures. The developed prediction methods and used performance models support Intel, AMD and multiple ARM-based multi-core architectures, six of which are presented and validated in this work. While GPGPUs are similar, they are out of the scope of this work. Four main stand-alone software components were developed: Kerncraft, generalized Layer Conditions (gLC), pycachesim and the Open Source Architecture Code Analyzer (OSACA). In the Kerncraft tool, static high-level code analysis is used to bring together individual data transfer and instruction throughput predictions into an explanatory comprehensive runtime and performance prediction, using the Execution-Cache-Memory and Roofline models. The built-in measurement functionality supports model validation and kernel benchmarking. The generalized Layer Condition model analysis gives an analytical closed-form cache behavior model for stencil-like kernels, which is used to predict cache utilization. The pycachesim cache simulator predicts cache behavior in more complex cases and for arbitrary kernels. OSACA comes with x86 and AArch64 support and can analyze the block throughput, critical path and loop-carried dependency.
Execution modeling (OSACA), cache modeling (pycachesim and gLC) and automatic performance modeling (Kerncraft) are evaluated and discussed using extensive validated studies, going far beyond what is practical to manually model. The presented research results reveal performance effects that were not classified, modeled or even known before. This lays the groundwork for further architecture performance research and model refinements, deepening our understanding of performance behavior on modern architectures. The published open source projects are already used and integrated with other research projects and software development utilities, indicating the relevance of this work beyond the academic fields of performance modeling and architecture analysis.
Autonomous driving is, besides electrification, currently one of the most competitive areas in the automotive industry. A substantially challenging part of autonomous vehicles is the reliable perception of the driving environment. This dissertation is concerned with improving the sensor-based perception and classification of the static driving environment. In particular, it focuses on recognizing road boundaries and obstacles on the road, which is indispensable for collision-free automated driving. Moreover, an exact perception of static road infrastructure is essential for accurate localization in a priorly built highly precise navigation map, which is currently commonly used to extend the environment model beyond the limited range of sensors. The first contribution is concerned with environment sensors with a narrow vertical field of view, which frequently fail to detect obstacles with a small vertical extent from close range. As an inverse beam sensor model infers free space if there is no measurement, those obstacles are deleted from an occupancy grid even though they have been observed in past measurements. The approach presented here explicitly models those errors using multiple hypotheses in an evidential grid mapping framework that neither requires a classification nor a height of obstacles. Furthermore, the grid mapping framework, which usually assumes mutually independent cells, is extended to information from neighboring cells. The evaluation in several freeway scenarios and a challenging scene with a boom barrier shows that the proposed method is superior to evidential grid mapping with an inverse beam sensor model. In the second contribution, a common shortcoming of occupancy grid mapping is approached. Multi-sensor fusion algorithms, such as a Kalman filter, usually perform obstacle association and gating to improve the obstacle position if multiple sensors detected it. However, this strategy is not common in occupancy grid fusion. In this dissertation, an efficient method to associate obstacles across sensor grids is proposed. Imprecise sensors are discounted locally in cells where a more accurate sensor detected the same obstacle and derived free space. The quantitative evaluation with an exact navigation map shows an increased obstacle position accuracy compared to standard evidential occupancy grid mapping. Whereas the first two contributions were concerned with multi-sensor fusion approaches for collision avoidance, the third uses occupancy grids for situation interpretation. In particular, this work proposes to use occupancy maps to classify the driving environment into the categories freeway, country or rural road, parking area, and city street. Identifying the current road type is essential for autonomous driving systems designed for limited environment types. Inspired by the success of deep learning approaches in image classification, end-to-end Convolutional Neural Networks are compared to Support Vector Machines trained on hand-crafted features. Two novel CNN architectures for occupancy-grid-based situation classification, designed for embedded applications with limited computing power are proposed. Furthermore, the occupancy-grid-based classification is fused with camera-image-based classification, and the road type probabilities are recursively estimated over time with a Bayes filter. The evaluation of the approaches on an extensive data set yielded accuracies of up to 99 %.
Measurement of the tissue sodium concentration (TSC) in human tissue offers important insights into pathological- and biological processes with applications in nephrology, neurology and myology. The TSC can be determined non-invasively by sodium magnetic resonance imaging (23Na MRI). The aim of this work is to accelerate 23Na MRI for TSC measurements in the skeletal muscle. Reconstruction methods and adapted acquisition techniques were investigated to reduce the acquisition time for 23Na MRI. The time saved could improve the applicability of TSC measurements for the use in the clinical environment and clinical studies. The main focus of this thesis is on compressed sensing (CS) reconstruction techniques and anisotropic acquisition techniques with reduced undersampling artifacts.
Here, the acquisition time reduction is achieved by undersampling, which means measuring less data points than needed for an image with a certain resolution. This undersampling causes artifacts and decreases image quality. The acquisition sequences and reconstruction techniques used and developed in this thesis seek to remedy these deficiencies.
First, 3D dictionary-learning compressed sensing (3D-DLCS) was evaluated for reconstruction of undersampled 23Na MRI data acquired with a density-adapted 3D-radial sampling sequence with a cuboid field-of-view (DA-3DPR-C). The results of the study with simulations and in vivo measurements indicated that 3D-DLCS could be applicable for a four-fold acceleration of TSC measurements in the skeletal muscle. However, the quantification error decrease was only about 10% compared to the reference.
Based on these results, optimized acquisition techniques were investigated and developed for accelerated 23Na MRI. Two acquisition sequences were used as a basis: the DA-3DPR-C method and a 3D acquisition-weighted density-adapted stack-of-stars sequence (AWSOSt). Two further acquisition sequences were developed to increase the artifact incoherence: A golden ratio rotated stack-of-stars sequence based on AWSOSt and a DA-3DPR-C adaption with golden means reordered projections. Comparisons in simulations and an in vivo/vitro study showed that the quantification error is decreased and the artifact incoherence is increased by adapting acquisition trajectories using the golden angle and golden means.
Additionally, three CS algorithms were compared for the adapted stack-of-stars acquisition sequence in simulations and an in vivo/vitro study: 3D-DLCS, total variation CS (TV-CS) and TV-CS with a block matching prior. In this comparison, the highest quantification error decrease of 35% was observed with 3D-DLCS for an acceleration factor of 4.1 and a measurement time of less than two minutes. The results demonstrate the impact of artifact incoherence for CS reconstruction. Furthermore, the investigations showed that applicable acquisition times are achievable by artifact-reduced acquisition and CS reconstruction.
Beyond CS, deep learning (DL) based reconstruction was explored in a preliminary study. A U-Net was trained using 350 23Na MRI calf measurements. The quantification error was decreased by 64% with the U-Net compared to the reference method. These promising results indicate that DL-based reconstruction could probably be the next step for accelerated 23Na MRI.
Zur Untersuchung der Bindenaht beim Spritzgießen von duroplastischen Formmassen auf Epoxidharzbasis
(2022)
Epoxidharze gehören zur Werkstoffgruppe der Duroplaste, die beispielweise unter Erwärmung aushärten und dadurch eine stabile Form erhalten. Aufgrund ausgezeichneter elektrischer Isolation und hoher Temperatur- und Medienbeständigkeit werden Epoxidharze oft in der Elektrotechnik- und Elektronikindustrie eingesetzt und gewinnen durch die Elektrifizierung der Mobilität immer mehr Aufmerksamkeit. Beim Spritzguss von Epoxidharzformmassen können bedingt durch die vorgegebene Bauteilgeometrie beim Zusammenfügen zwei getrennter Fließströmungen Bindenähte entstehen. Diese Bindenähte resultieren oft mechanische Schwachstellen und sind als limitierter Faktor im gesamten Bauteil bekannt. Da duroplastische Formmassen durch Härtungsreaktion eine Formstabilität im Spritzgießprozess erhalten, wird die Verbundfestigkeit an der Bindenaht durch chemische Reaktion bzw. kovalente Bindungen zwischen beiden Strömungen erzeugt. Je nach Prozessführung können Formmassen vor der Entstehung der Bindenaht unterschiedlich vorvernetzt sein. Aktuell gibt es noch keine klare Antwort auf die Frage, ob die Vorvernetzung in der Epoxidharzformmasse für die verminderte Bindenahtfestigkeit verantwortlich ist. Darüber hinaus sind spritzgießfähige duroplastische Formmassen immer mit diversen Füllstoffen gefüllt. Es fehlen Untersuchungen für das Verständnis, wie Füllstoffe von Epoxidformmassen sich auf das Fließverhalten im Spritzguss auswirken und anschließend die Bindenahtfestigkeit beeinflussen.
Im Rahmen dieser Arbeit soll die Bindenahtfestigkeit beim Spritzguss von Epoxidformmassen im Zusammenhang mit dem Reaktionsablauf beim Formfüllen sowie dem durch Füllstoffe resultierenden Fließverhalten erklärt werden. Dabei werden ausgewählte prozesstechnische und werkstoffliche Parameter im Spritzguss variiert und die resultierenden Material- und Bauteileigenschaften eingehend charakterisiert. Ziel der Arbeit ist es, die grundlegenden Ursachen für eine verminderte Bindenahtfestigkeit im Spritzguss von Epoxidharzformmassen zu verstehen. Damit können Gegenmaßnahmen zur Optimierung der Produktqualität beim Einsatz der Materialgruppe gezielt abgeleitet werden.
Every day, organizations following a collect-everything mentality generate, process, and store an ever-increasing amount of data. With the increasing amount of data available, it is becoming more difficult and complex to analyze it in a way that creates beneficial knowledge. This analysis is usually done via queries, i. e. the queries themselves also contain knowledge in turn.
The goal of this dissertation is to make the knowledge contained in the queries available. Often, this knowledge is not immediately apparent but is tacit expert knowledge that usually exists only in the heads of a few domain experts. As a solution, this dissertation presents QRep, a query repository that can manage SQL queries and the knowledge they contain in the form of query metadata. The functionality provided by QRep goes far beyond that of a conventional database system. It can derive multiple query metadata automatically. For metadata that do not allow this, it provides a mechanism to semi-automatically derive the underlying
query knowledge via evolutionary domain-specific policy rules.
This dissertation first provides a conceptual data model for the management of query metadata. Second, it describes the mapping of this schema partly to a relational schema and a multi-relational directed property graph. To enable a performant retrieval of query metadata, QRep internally accesses tree-structured metadata via
graph traversals and tabular metadata via SQL. For simple, uniform user access to the query metadata, QRep provides a domain-specific language that can be easily adjusted and extended. Its fundamental parts are so-called basic query-processing patterns, which can be nested and combined via Boolean operators to form arbitrary query-processing patterns.
The domain-specific language is one component of the policy rules. These rules enable the externalization of tacit expert knowledge regarding queries. Furthermore, they do not require profound technical knowledge and can be defined evolutionarily at runtime. A policy rule is based on a conditional rule and automatically classifies queries matching a processing pattern according to its consequent part. For this, QRep incrementally aligns the basic patterns in the policy rule with the metadata of the query and stores the classification result as additional metadata if this alignment evaluates to True.
The pilot implementation of QRep features a conventional three-tier client-server architecture and provides a user-friendly graphical user interface. It can be deployed in a minimally-intrusive way to arbitrary existing IT landscapes to facilitate integration
into existing projects. The average latency for deriving metadata for a query is 366 ms, and 906 ms for aligning a query with all policy rules. For long-running queries, these latencies are negligible compared to their execution time.
Log files are plain text files written by any modern computer system. Their content is determined by logging statements during software development. Thus, any kind of event can be collected during runtime. As a result, logs contain a wealth of precious information that is used for various analyses. However, current log processing methods assume static log structures and contents. Due to today’s focus on agile software development paradigms, source code and therein contained logging statements can constantly change. This leads to failing preprocessing and incorrect data points for subsequent analyses. If those failures are not detected,
faulty information will be extracted which will result in erroneous insights.
This dissertation targets successful log file processing from different angles using artificial intelligence approaches. First, missing data points induced by changing software are addressed. Using Magnetic Resonance Imaging (MRI) examination data as an example, new variables are added to the logging mechanism with
growing requirements. As a consequence, logs from systems with earlier software version are missing the new features. Therefore, we propose classification techniques to learn feature correlations from complete data sets. Afterwards, these trained models are applied to impute data points where the respective feature is
missing. We prove the effectiveness of a feed forward neural network by successfully determining the desired feature for more than 94% of the MRI exams.
Furthermore, missing information can also originate from failing parsers. Despite availability within the raw logs, crucial data points might not be extracted correctly. Since software changes can entail log file structure adaptions, they can pose insurmountable challenges to existing log parsing methods. We investigate Hidden Markov Models and Deep Learning methods in order to process log files automatically and reliably despite those challenges. We study different log file data sets from various systems with manifold log changes and outperform state-of-the-art parsers. Thus, we propose a novel pipeline for flexible parsing. It contains a stateful Long Short-Term Memory (LSTM) network to model adaptability regarding log changes. We call it FlexParser and yield for all studied data sets an F1-Score of 92% and higher.
Successfully parsed log files hold manifold chances to find actionable insights. Therefore, we process the parsing results of MRI log files in order to predict hardware failures. High image quality is crucial for medical diagnosis, however, directly depends next to the selected imaging parameters also on flawless hardware. Following, we train Deep Learning models on image features and respective recorded hardware conditions. Since available data points from failing components is limited, we employ different data augmentation techniques. Furthermore, we investigate Ensemble Learning to combine insights from different models and compare results with those achieved by time series methods. Concluding,
we propose an Ensemble Learning pipeline which reliably detects hardware failures achieving an F1-Score of 94.14%.
With the introduction of computer-aided methods for diagnosis and intervention, patient outcomes of many clinical procedures have been increased tremendously in the last decades. An essential task thereby is the registration of inter- or intrapatient images that are acquired using a single or multiple imaging modalities. In principle, vasculature permeates through all organs of human body. As it is interbedded spatially, it reflects the pathological changes of the surrounding tissue. Thus, accurate and robust registration methods to align vessel structures or images have the capability to benefit a large variety of clinical procedures.
Numerous methods that propose novel vessel registration algorithms have been published to date. Both intensity and feature-based methods are proposed, where a clear dominance of feature-based, especially point-based methods, can be derived from the summary of the state-of-the-art. Although the methods proposed in the literature demonstrate promising results for dedicated clinical set up, a generalized approach for vessel registration regardless of region of interest has not been proposed and evaluated. Furthermore, most state-of-the-art methods requires cumbersome hyperparameter tuning, which reduce their clinical applicability consequently. Therefore, a primary goal of this thesis is to develop, implement and evaluate an accurate and inherent robust vessel registration framework with the generalizability and applicability to various clinical applications. Recent progress in machine learning and deep learning opens new perspectives to improve the efficiency of the conventional registration methods for vasculature. Promising registration results have been achieved with different learning paradigms, such as reinforcement learning, supervised- and unsupervised learning with synthetic and clinical data. Hence, a secondary goal of this thesis is to investigate the potential of other machine learning and deep learning techniques to resolve vessel image registration problems.
One of the two point-based registration frameworks investigated in this thesis utilizes mixture models to align the centerlines of vasculature. Thereby, a hybrid mixture model is proposed as a key part of the framework, that models the spatial and topological information of the vasculature simultaneously and is consequently equipped with significant discriminative capacity. Moreover, an automatic refinement mechanism to identify regions with missing data is formulated. The final transformation to the target image can be estimated with different methods such as Thin-Plate-Spline, B-Spline or Gaussian Process. The evaluation with synthetic, phantom and clinical data acquired from different clinical procedures demonstrate the accuracy and inherent robustness of the entire vessel registration framework regardless of clinical set up.
The other approach formulated in this thesis makes use of imitation learning paradigm to overcome the weakness and challenges of adopting reinforcement learning for image registration problem. Under the guidance of a demonstrator an agent is trained to find the optimal displacement of landmarks. The network architecture is inspired by PointNet that is able to consume raw point data as input. The proposed framework is evaluated with clinical fundoscopic images in a retrospective manner. A special attention of experiments conducted is to understand the principles of our imitation network, where the influences of the model parameters are analyzed in detail. The evaluation results demonstrate the potentials and effectiveness of the imitation network to register vascular images for the first time.
With the ongoing development of mobile and desktop operating systems, existing file systems also get updated with new features, or the vendors introduce even new file systems. Since Android 7.0, the predominant full disk encryption gets step-by-step replaced by the new file-based encryption (FBE), implemented as an ext4 feature. Since Android 10, this disk encryption scheme gets mandatory for all new Android devices.
On the side of desktop and server OSs, Microsoft introduced an all-new file system called Resilient File System (ReFS) which should replace the NFTS in the long run. Starting as a file system for servers, ReFS introduces new features to make data storage more robust and efficient.
This work investigates the new technologies ext4 FBE and ReFS in several aspects of forensic data extraction. We investigate the amount of information leakage through unencrypted metadata in Android’s FBE. We propose a generic method and provide appropriate tooling to reconstruct forensic events on Android smartphones encrypted with FBE, which require no knowledge of the encryption key. Based on a dataset of 3903 applications, we show that files’ metadata can be used to reconstruct the name, version, and installation date of all installed apps. Furthermore, based on WhatsApp, we show that information leakages through metadata can even be used to reconstruct a user’s behavior depending on a specific app.
To furher enhance the forensic data extraction of FBE encrypted Android disks, given a raw memory image, we present a new encryption key recovery method tailored for FBE. Furthermore, we extend The Sleuth Kit to automatically decrypt file names and file contents when working on FBE-enabled ext4 images, as well as the Plaso framework to extract events from encrypted ext4 partitions. Last but not least, we argue that the recovery of master keys from FBE partitions was straightforward due to a flaw in the encryption key derivation method by Google.
On server and desktop systems, Microsoft has released ReFS, that internal structures are not officially documented. Therefore we reverse-engineered these internal structures and behavior and documented them. Based on these structures and the access processes that modify them, approaches to recover (deleted) files and older states from ReFS formatted partitions are shown. We also evaluate our implementation and the allocation strategy of the ReFS driver concerning the accuracy, runtime, and the ability to recover older file states.
At last, with the knowledge of the internal ReFS structures and the threat of flaws in forensic software in mind, we implemented a structure-aware coverage guided fuzzy testing framework explicitly tailored to ReFS to find undetected security-critical flaws. With the new complex features of ReFS, the driver is growing more extensive and more complex, increasing the attack surface of the Windows kernel. Attackers can often use security-critical bugs in file system drivers to escalate privileges by mounting a well-prepared file system. Such an attack is also relevant in forensic data extraction because criminals can use it to prepare malicious disks, which will hamper or completely circumvent the extraction process by compromising the analysis environment.
We demonstrate the effectiveness of our fuzzing approach by finding 27 unique payloads that panic the Windows kernel when mounting or accessing ReFS partitions. Microsoft confirmed those bugs and acknowledged eight unique CVEs which allow remote code execution attacks.
With our overall findings, the forensic community should be well prepared for extracting data from modern file systems used by mobile and desktop systems. With the help of the new proposed fuzzy testing framework, forensic software can be hardened against severe anti-forensic methods by patching the discovered flaws.
Proceedings of the 2022 Joint Workshop of the German Research Training Groups in Computer Science
(2022)
Having spent two successive years running online to prevent the spread of the
Corona virus, the traditional annual meeting of the German Research
Training Groups (RTGs) funded by the Deutsche Forschungsgemeinschaft
(DFG) in the field of computer science returns to Schloss Dagstuhl --–
Leibniz Center for Informatics, one of the world's premier venues for
computer science-related seminars. Returning to Dagstuhl and hosting
this meeting as an in-person-only event was a deliberate decision to
revive interaction modes that many of the funded researchers had yet
to experience: fostering personal interchange of ideas and experiences
in order to strengthen the connection within the German computer
science community. This volume documents the abstracts of the
research topics of funded researchers in the participating RTGs.
The event was jointly organized by RTG 2475 (Cybercrime and
Forensic Computing) and RTG 2428 (ConVeY --- Continuous Verification
of Cyber-Physical Systems). It took place between Sunday, June 12 and
Wednesday, June 15, 2022, as in-person only Dagstuhl Event
22243. The meeting featured the usual sequence of research
presentations by funded researchers, networking meetings for PIs and
RTG coordinators, as well as two invited talks, one by Professor
Martina Seidl (JKU Linz, Austria) on ``Competitions as Scientific
Method'' and another by Professor Jennifer Byrne (School of Medical
Sciences, The University of Sydney, Australia) titled ``An
introduction to research paper mills''.
Because last year's event marked the 25th anniversary of the workshop
series, it featured a live interview with Professor Otto Spaniol who
had initiated the workshop series in 1996. We document the interview
in this volume.
In the context of the United Nations Paris Climate Agreement of 2016, the majority of the global leading automotive manufacturers are committed to electrifying their fleets. A particular challenge in achieving this transformation is the efficient and economical development of new types of battery systems to meet the high customer requirements for electric range and fast-charging capability as well as legally required safety standards. These requirements must be guaranteed over the entire vehicle lifetime. However, the battery ages over time due to electrochemical degradation effects during operation. As a consequence, the battery state needs to be continuously monitored and analyzed. Thereby, new ways of analysis are required, as the current characterization of the battery state during maintenance is associated with high financial efforts, time-consuming measurement procedures, and is limited to a low number of available test capacities.
An innovative and scalable alternative is offered by deploying battery models. In this context, this thesis addresses the research question to what extent battery-electric modeling is applicable to determine the battery state, using only in-vehicle operating data. To this end, this approach is divided into two research areas: First, the modeling of current electric battery behavior based on in-vehicle data, and second, the methodology for analyzing the battery state.
While conventional battery models are mainly based on physical system representations, this thesis focuses on novel data-driven methods that are able to independently learn relevant correlations from vehicle operational data and to use this information to continuously update the battery model. In a preliminary analysis, artificial neural networks with a sliding window approach proved to be a suitable candidate to learn the electric battery behavior during operation.
In terms of the methodology, the analysis of the battery state is considered separately at cell-level and system-level due to the high complexity of the battery behavior and the possible operating conditions inside electric vehicles. The development and evaluation of the methodology for battery state determination are carried out at cell-level by using extensive operating and testing conditions. In particular, pulse tests and incremental capacity analysis achieved high accuracies. However, the results of this work also show that the cell-level methodology cannot be directly extrapolated to system-level battery behavior due to systematic and statistical uncertainties.
Apart from the current limitations of neural battery models, various optimization potentials in the area of the training process and data preprocessing are identified. In conclusion, the obtained findings provide an outlook for further applications in the context of data-driven battery analysis.