006 Spezielle Computerverfahren
Refine
Year of publication
Document Type
- Doctoral Thesis (70)
- Article (4)
- Moving Images (3)
- Master's Thesis (2)
- Report (1)
Has Fulltext
- yes (80)
Keywords
- Maschinelles Lernen (14)
- Beschleunigungssensor (7)
- Ingenieurwissenschaften (6)
- Maschinenbau (6)
- Mustererkennung (6)
- Produktionstechnik (6)
- Ganganalyse (5)
- Hochleistungsrechnen (5)
- machine learning (5)
- pattern recognition (5)
Institute
- Technische Fakultät (42)
- Department Informatik (24)
- Department Maschinenbau (6)
- Lehrstuhl für Konstruktionstechnik (KTmfk) (4)
- Department Elektrotechnik-Elektronik-Informationstechnik (3)
- Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik (FAPS) (2)
- Naturwissenschaftliche Fakultät (2)
- Fachbereich Wirtschaftswissenschaften (1)
- Institut für Medizininformatik, Biometrie und Epidemiologie (1)
- Philosophische Fakultät und Fachbereich Theologie (1)
Single instrument recognition is a research topic that is worked on for many years. After the AI winter the approaches switched from feature based methods to artificial neural networks. When the recognition rates increased the community moved on to harder problems, at the latest when the community announced that the single instrument recognition is solved.
In this thesis I am going to show that the assumptions that are the basis for this statement are erroneous. The main reason is the misbelief of the comparability of accuracies. To approach this problem I will introduce an evaluation method that is less prone to external factors and a well obtainable data base that enables a better comparison of algorithms.
To evaluate the actual level of the resolving of single instrument recognition and to push this frontier, I refined, combined, and evaluated the feature and the artificial neural network based approach. During this process a new feature was developed and well known features were improved, and artificial neural network architectures were optimized and compared by applying evolutionary programming.
Up to today telephone speech lacks of perceptual quality and in- telligibility due to bandwidth removal and quantisation artefacts in the encoding process. Blind bandwidth extension artificially regenerates this missing frequency content and thus improves the perceptual quality and intelligibility.
This work proposes novel approaches for blind bandwidth extension based on generative deep neural networks with convo- lutional or recurrent architectures. Motivated by the source-filter model of the human speech production, two of the proposed systems decompose the speech signal into spectral envelope and excitation signal. Each are bandwidth-extended with ded- icated networks. First the envelope is extrapolated and then the network extrapolating the excitation signal is trained such that there is no mismatch between the excitation signal and the envelope. By this, we achieve better perceptual quality with lower computational complexity compared to a system that ex- tends the bandwidth of the undecomposed speech signal with a single network. Input and output of all proposed systems is raw time-domain speech without any preprocessing. They are trained with a mixture of adversarial and perceptual loss which further improves the perceptual quality. To avoid mode collapse and a more stable adversarial training, spectral normalisation is employed in the discriminator.
The presented systems are compared to previously published systems by objective measures and subjectively by a listening test. An estimate of the computational complexity is given and compared to state of the art speech coding technologies. Objec- tive and subjective tests show that the proposed systems deliver substantial better quality than prior techniques. It was further shown that our systems reduces the word error rate of a speech recognition system.
Mobile communication has become ubiquitous at the beginning of the 21st century,
facilitating technological revolutions like Industry 4.0 or the Internet of Things (IoT).
Mobile devices are often battery-powered and, thus, tightly energy-constrained. To
sustain the skyrocketing number of devices and performance requirements of indi-
vidual applications in the years to come, elaborate techniques to reduce the energy
consumption of such devices are thus required. Apart from technological advances
that lower the power consumption of mobile device modems regardless of their opera-
tional mode, Dynamic Power Management (DPM) comprises techniques to reduce
the energy consumption of mobile device modems by turning idle components to low
power-consuming states whenever possible.
However, state-of-the-art DPM techniques are reactive rather than predictive and
these reactive techniques observe information too late to take full advantage of periods
of transmission inactivity. As a remedy, this dissertation presents predictive DPM
techniques that anticipate periods of transmission inactivity to allow the modem to
enter low power-consuming states earlier and enter deeper sleep states consuming even
less power. This reduces energy consumption without missing any information—in
the case of correct predictions.
Unfortunately, a predictor cannot always perform correct predictions. Prediction
errors may degrade the data rate because missed grants have to be re-transmitted.
Due to this, the predictive DPM technique must be designed to minimize the number
of erroneous predictions and needs to perform training whenever the prediction is
not accurate enough. Additionally, for predictive DPM to achieve energy savings,
the energy consumption overhead caused by the prediction algorithm must be lower
than the amount of energy saved by entering low power-consuming states earlier and
entering deeper sleep states. Taking these considerations into account, this dissertation
proposes a novel predictive DPM methodology with a minimal degradation of the
data rate due to mispredictions and the goal to achieve maximal achievable energy
savings—under realistic conditions.
In the current mobile communication standards Long Term Evolution (LTE) and
5th Generation New Radio (5G NR), base stations schedule all traffic sent to and
from mobile devices using grants contained in a control channel, aligned to so-called
Transmission Time Intervals (TTIs). State-of-the-art reactive DPM behavior of User
Equipments (UEs) is to decode the control channel blindly, even though the control
channel often contains no grants for the specific UE. As a result, a UE modem
potentially wastes precious energy to decode the control channel only to find that there
was no grant for the UE contained.
This dissertation makes the following contributions in the context of predictive
DPM. A prediction is performed before the control channel is decoded to infer whether
it will actually contain a grant or not. This allows a mobile device to decode the control
channel only when a grant presence is predicted. Control channel decoding is skipped
whenever a grant absence is predicted, and modem components immediately enter low
power states.
As a first contribution, an adaptive predictive DPM technique is presented for the
protocol standard LTE that is designed for situations where no a priori knowledge of
grant patterns is available. This implies that predictive DPM first has to be trained
online, where an energy consumption overhead is incurred (for training), but no
energy can be saved. This requires a low energy consumption overhead for prediction
and mechanisms that monitor and minimize prediction errors. In this context, an
adaptive control strategy is presented that lets predictive DPM make a suitable trade-
off between exploration and exploitation, deciding at run time when the prediction
is reliable enough to choose the proposed predictive policies (exploitation), and
when the predictive DPM should rather perform online learning (exploration) before
subsequently returning to exploitation.
The assumption of having no a priori knowledge is a worst-case assumption, as
usually, a large number of known traces is available. Thus, the second contribution is
given by a trace-aware predictive DPM technique, which leverages known traces to
shift as much predictor training as possible from energy-constrained run time (online)
to design time (offline). Whenever a trace similar to the ones used during offline
training is encountered, trace-aware adaptive predictive DPM can forego exploration
and enter exploitation—and thereby reduce energy consumption—immediately. This
eliminates the need for online learning for known traces, and thus reduces the energy
consumption overhead that would be necessary for training.
The third contribution considers the application of deep-learning algorithms that can
be used to forecast not only a single TTI, as investigated in the prior contributions, but a
forecast period of multiple TTIs. On the one hand, such forecasts significantly increase
the energy consumption overhead of the prediction algorithm. On the other hand,
they enable predictive DPM to initiate longer and deeper sleep states when multiple
consecutive grant-free TTIs are predicted. Multiple deep learning algorithms and
the impact of different forecast periods on prediction accuracy as well as achievable
energy savings are evaluated.
The prior contributions were initially designed and optimized for the LTE protocol
standard. During the research for this dissertation, the 5G NR standard was released,
which is a major evolution of LTE. While control signaling concepts stayed similar,
changes to give more flexibility for different service classes—from ultra-reliable
low latency communications (URLLC) over massive machine-type communications
(mMTC) to enhanced mobile broadband (eMBB) were introduced.
In particular, 5G NR introduced the so-called cross-slot and the multi-slot scheduling
mechanisms. As a forth contribution of this dissertation, solutions for predictive DPM
are presented that also exploit these extensions to reduce the energy consumption of
modems even further.
Projection mapping is a form of augmented reality, where virtual content is brought into the real world by means of one or multiple projectors illuminating a scene. This allows to completely alter the visual appearance of possibly moving real-world objects without the need of wearable devices, such as smartphones or head-mounted glasses. Projection mapping therefore scales seamlessly to large audiences and thus becomes
increasingly popular in art, education, product design and more.
However, since the content is situated on real-world surfaces, various challenges have to be overcome in order to achieve a truly immersive and captivating experience. For example, the parameters of the entire system, such as the position and orientation of each projector, must be calibrated to extraordinary accuracy. Even small deviations in the estimation of these parameters result in visible errors, where the projected content does not coincide with the real target object geometry.
In contrast to other forms of augmented reality, projected content is further strongly impacted by external light sources, the target's surface color and the projectors' color reproduction capabilities. Disregarding these effects may result in disparities between the artist's intent and the final projection.
Lastly, in order to cover large surface areas with possibly complex geometric features, modern projection mapping systems rely on large numbers of projectors. The projection thus varies in brightness depending on the number of projectors contributing to a surface point's illumination. This effect must be compensated in software, taking both surface properties and the projected content into account.
The described issues are amplified if commodity hardware is employed and the setup is not perfectly isolated from outside influences like user interaction and heat. Solutions to these issues must therefore be robust to inaccuracies and limitations of consumer-grade components and changing lighting conditions.
In this doctoral theses we propose novel solutions to the problems outlined above, which enhance the quality and usability of projection mapping compared to previous approaches tackling the same issues. We present an automatic calibration routine, which accurately determines the parameters of the real-world setup. Our adaptive color correction algorithm determines the environment light, surface color and projector
response and corrects each image in real-time, taking the limited range of displayable colors and human color vision into account. Lastly, we propose a novel content-aware brightness solver, which seamlessly balances the intensity of any number of projectors in large projection mapping systems and mitigates the effect of small remaining calibration errors.
We publish a free open-source projection mapping system, which allows others to
compare their results to ours and further build on our algorithms.
One of the main goals in the development of audio signal coding and processing algorithms is to achieve the best possible sound quality within a given set of design parameters (e.g., bitrate and computational complexity constraints). The quality of an audio processing system is mostly relevant from a perceptual point of view: it is closely connected to the impression that processed signals by said system will elicit on a human listener. Perceived overall sound quality is one of the most important factors for market acceptance of a digital audio codec. Considering this fact, a reliable quality assessment method is of utmost importance.
Perceptual audio coding algorithms can significantly increase their signal coding efficiency by eliminating information that is deemed to be redundant and irrelevant to the listener. However, the coded signals can contain noticeable artifacts that impair the signal and thus degrade the resulting quality. Therefore, a good quality assessment method must consider the role of such artifacts in the perception of quality degradation. Audio coding artifacts have been described as noisiness, sound coloration/dullness and roughness among other classes of degradations in the case of sound timbre quality. In the case of perceptually coded stereo or multi-channel audio signals, disturbances in the evoked spatial image --- such as loss of envelopment or auditory object localization distortions --- can occur. Additionally, quality degradation perception is not the same in low-quality signals (i.e., where multiple artifact types cause large impairments at the same time) as in high-quality signals (very few artifacts are perceived, causing small impairments). These factors make quality assessment a considerably complex task.
Due to the mentioned complexity of quality assessment, performing carefully conducted listening tests on many subjects is often considered as the gold standard for quality assessment methods in audio coding. Quality evaluation procedures based on listening subject responses are usually referred to as subjective quality assessment methods. However, subjective methods need a considerable amount of time and resources for their design, execution and response analysis. As a result, the last twenty years have seen the advent of computer-based analysis systems applied to the coded signals that produce quality estimates attempting to predict average subject responses in quality rating tasks. These computer-based analysis procedures are known as objective quality assessment methods and they aim to reduce the time and costs associated with subjective methods. Inspired by their success in the field of audio coding, such measurement systems also make use of perceptual auditory models to develop meaningful predictors of sound quality. This cost-effective alternative to listening tests is especially useful in everyday codec development where new perceptual coding algorithms are constantly tested. As an additional advantage, quality measurement algorithms can be built in perceptual audio codecs and provide quality predictions (e.g., in a feedback loop) at encoding time, so that the codec can adapt its parameters to increase the resulting coded signal quality. Still, many of the existing quality measurement systems only reliably predict quality degradation for a very restricted set of signal artifacts (e.g., some aspects of timbre quality only), or for limited quality ranges (e.g., signals with single small impairments). These limitations in quality prediction have been linked to particular limitations in the underlying perceptual models of the measurement systems.
This thesis proposes several contributions to improve timbre and spatial aspects of objective quality assessment methods by extending their perceptual models. Firstly, considering the timbre aspects of quality prediction, this thesis proposes a novel paradigm to model the interactions of the different perceived distortion artifacts at a cognitive level. When this paradigm is applied to an existing quality measurement system, the extended system predicts subjective responses of a wide variety of listening test databases with a greater accuracy than a series of previously proposed quality measurement systems. Particularly, the interaction model allows the extended quality measurement system to predict subject responses more accurately than previously proposed solutions at wider quality ranges (low quality signals and high quality signals). Secondly, the same existing quality measurement system is extended to increase assessment performance of coded signals with spatial distortions. This is done by extending its perceptual model so that it considers binaural aspects of hearing. Particularly the concept of disturbance loudness, a widely-adopted concept in timbre quality models, is extended to account for spatial aspects.
Fluid flows that transport rigid particles are ubiquitous in Nature and industrial processes where they exhibit a rich set of fascinating and complex dynamics.
Developing a thorough understanding of these particle-laden flows is mainly based on studies at the microscale, where relatively small but representative systems are investigated in detail.
Corresponding simulations that aim to assist this procedure are challenging due to the vast computational costs involved in accurately resolving all relevant physical effects.
Their successful usage necessitates efficient algorithms designed for high-performance computing and extensive verification studies to ascertain the simulation's broad applicability and correctness.
This thesis develops a coupled fluid-particle simulation method that tackles these challenges and demonstrates its usability for such fully resolved simulations at the microscale.
To this end, we combine the lattice Boltzmann method with the discrete element method in the open-source framework waLBerla.
While the former method is used to efficiently represent the fluid flow dynamics,
the discrete element method accounts for frictional collisions between the rigid particles.
Their coupling is based on the momentum exchange between the fluid and the particle phase.
We provide a detailed description of the algorithmic components and all involved model parameters.
Focusing on the characteristics of the applied coupling approach, we present and compare several techniques to improve the simulation's accuracy and stability.
Measures include higher-order boundary conditions, temporal averaging of interaction quantities, and adaptions of the lattice Boltzmann method.
We assemble an extensive calibration and validation pipeline for all relevant aspects of dense particulate flows, universally applicable for such simulation approaches.
This concept is utilized to gain valuable insight into our coupled algorithm's behavior and derive guidelines for the model parameterization.
In particular, explicitly considering short-range hydrodynamic interactions between particles and an adequate temporal resolution of single collision events are found to be essential ingredients to obtain accurate collision dynamics of submerged particles.
The method's algorithmic components work on strictly localized subsets of the complete data.
This characteristic feature is a prerequisite for efficient usage of the immense compute power offered by today's supercomputers.
In many cases, the domain sizes and runtimes necessary for simulative microscale studies can only be reached through such massively parallel execution with thousands of processes.
We explore the possibilities and limitations of our unified parallelization strategy via scaling experiments on up to 65536 processes.
As fluid-particle simulations often exhibit a spatially and temporally varying workload, we show that performance improvements can be achieved via dynamic load balancing techniques.
They quantify current imbalances between the processes and then re-distribute the workload.
We finally illustrate the predictive capabilities of our method by investigating the erosion and transport of densely packed sediment beds consisting of several ten thousand particles with vastly different sizes.
The detailed information obtained from such a simulation allows us to study phenomena like size-based vertical sorting and derive rheological descriptions of the macroscale system dynamics.
This thesis thus contributes a valuable building block in the multi-scale modeling framework of particulate flows.
Image enhancement and recognition are two important applications in computer vision and medical imaging. Current approaches to image enhancement and recognition work well under ideal conditions, but they struggle with low-quality images that are common in real-world applications. In such cases, image enhancement and recognition must account for the poor data quality to produce reliable results.
This thesis presents strategies to increase the robustness of image enhancement and recognition for low-quality images. We present techniques for model-based and learning-based approaches. Model-based approaches invert the image acquisition using iterative optimization. Learning-based approaches learn the task from data captured with the same image acquisition pipeline expected in the wild.
In the first part of the thesis, we present the novel adaptive quantile sparse image prior for robust image enhancement with a model-based approach. The prior is robust against challenging degradations due to its non-linearity and spatially adaptive weights. In extensive experiments, we validate that the prior is an appropriate model of natural images.
The second part is split into the two major topics of learning-based image enhancement and license plate recognition. We provide two strategies for each topic.
First, we present two strategies to improve the robustness of learning-based image enhancement techniques. One strategy evades the sensitivity of super-resolution towards unseen noise distribution by incorporating a denoiser into an end-to-end approach. We present two different architectural designs for the combination of denoising and super-resolution. The second strategy combines super-resolution with digit classification to circumvent choosing a suitable denoiser. In the proposed multi-task framework, the shared layers between both tasks act as a denoiser.
Second, we explore robust and reliable license plate recognition. First, we train a neural network on severely degraded images to increase robustness against compression and to explore its limitations. Second, we evaluate several probabilistic methods to anticipate failure cases based on the prediction uncertainty.
This work presents robust image enhancement and recognition approaches using super-resolution and license plate recognition as example applications. However, the methods are not limited to these and can be used, for example, in dehazing or face recognition in the wild.
Komplexe Produkte der fertigenden Industrie, wie z. B. im Fahrzeugbau, zeichnen sich durch eine hohe Variantenvielfalt aus und bestehen aus einer wachsenden Anzahl von Komponenten, die der Hersteller von Lieferanten beschaffen muss. In der frühen Phase der Produktentwicklung ist es wichtig, die Kosten dieser Fremdbezugskomponenten solide abzuschätzen, u. a. um Entwicklungsalternativen zu bewerten oder die Verhandlungen mit potenziellen Zulieferern zu unterstützen. Das sog. Cost Engineering im Supply Management eines Herstellerunternehmens übernimmt diese Aufgabe. Traditionelle Kalkulationsverfahren erweisen sich jedoch zunehmend als zu aufwändig, um die Beschaffungsvielfalt zu bewältigen.
Um diesem Problem zu begegnen, sind neuartige und möglichst intelligente Ansätze zu erarbeiten, die es ermöglichen, in der frühen Produktentwicklungsphase durch eine (Teil-) Automatisierung des Kalkulationsprozesses die Effizienz im Cost Engineering zu steigern.
In diesem Buch wird gezeigt, dass sich vielfältige Methoden aus dem Gebiet des maschinellen Lernens für eine intelligente Kostenprognose eignen und die Aufgaben des Cost Engineers wirkungsvoll erleichtern können. Dies wird zum einen durch theoretische Analysen des State of the Art fundiert und zum anderen durch differenzierte Ergebnisse von drei praktischen Fallstudien mit jeweils zahlreichen Einzelexperimenten belegt.
In diesen Fallstudien werden Lösungskonzepte erarbeitet und in der Unternehmenspraxis evaluiert, die einen Cost Engineer bei seinen Kostenschätzungen mithilfe von Machine Learning (ML) zielgerichtet unterstützen. Durch das experimentelle Design und eine Vielzahl von Testergebnissen gelingt es, Best Practices von ML-Modellen für die Kostenprognose von zu beschaffenden Bauteilen und Baugruppen herauszudestillieren. Ein derartiges ML-Modell ist insbesondere durch die „beste“ Kombination aus der immensen Vielfalt von ML-Algorithmen, zugehörigen Trainings- und Testkonzepten, Featureselektionsmethoden sowie Erklärungs-ansätzen der sog. Explainable Artificial Intelligence charakterisiert. Daneben werden wertvolle Erkenntnisse über die Leistungsfähigkeit und Genauigkeit von ML-Ansätzen gewonnen.
Ein abschließendes Kapitel zeigt im praktischen Umfeld bestehende Herausforderungen bei der Einführung derartiger Systeme zur intelligenten Kostenprognose auf. Hierzu dient eine empirische Untersuchung mit einer Befragung von 130 Managern in einschlägigen Verantwortungsbereichen. Aufbauend auf den gewonnenen Erkenntnissen werden schlussendlich Limitationen wie auch zukünftige Möglichkeiten des maschinellen Lernens für das Cost Engineering sowohl aus der wissenschaftlichen als auch aus der Praxisperspektive vorgestellt.
Autonomous driving is, besides electrification, currently one of the most competitive areas in the automotive industry. A substantially challenging part of autonomous vehicles is the reliable perception of the driving environment. This dissertation is concerned with improving the sensor-based perception and classification of the static driving environment. In particular, it focuses on recognizing road boundaries and obstacles on the road, which is indispensable for collision-free automated driving. Moreover, an exact perception of static road infrastructure is essential for accurate localization in a priorly built highly precise navigation map, which is currently commonly used to extend the environment model beyond the limited range of sensors. The first contribution is concerned with environment sensors with a narrow vertical field of view, which frequently fail to detect obstacles with a small vertical extent from close range. As an inverse beam sensor model infers free space if there is no measurement, those obstacles are deleted from an occupancy grid even though they have been observed in past measurements. The approach presented here explicitly models those errors using multiple hypotheses in an evidential grid mapping framework that neither requires a classification nor a height of obstacles. Furthermore, the grid mapping framework, which usually assumes mutually independent cells, is extended to information from neighboring cells. The evaluation in several freeway scenarios and a challenging scene with a boom barrier shows that the proposed method is superior to evidential grid mapping with an inverse beam sensor model. In the second contribution, a common shortcoming of occupancy grid mapping is approached. Multi-sensor fusion algorithms, such as a Kalman filter, usually perform obstacle association and gating to improve the obstacle position if multiple sensors detected it. However, this strategy is not common in occupancy grid fusion. In this dissertation, an efficient method to associate obstacles across sensor grids is proposed. Imprecise sensors are discounted locally in cells where a more accurate sensor detected the same obstacle and derived free space. The quantitative evaluation with an exact navigation map shows an increased obstacle position accuracy compared to standard evidential occupancy grid mapping. Whereas the first two contributions were concerned with multi-sensor fusion approaches for collision avoidance, the third uses occupancy grids for situation interpretation. In particular, this work proposes to use occupancy maps to classify the driving environment into the categories freeway, country or rural road, parking area, and city street. Identifying the current road type is essential for autonomous driving systems designed for limited environment types. Inspired by the success of deep learning approaches in image classification, end-to-end Convolutional Neural Networks are compared to Support Vector Machines trained on hand-crafted features. Two novel CNN architectures for occupancy-grid-based situation classification, designed for embedded applications with limited computing power are proposed. Furthermore, the occupancy-grid-based classification is fused with camera-image-based classification, and the road type probabilities are recursively estimated over time with a Bayes filter. The evaluation of the approaches on an extensive data set yielded accuracies of up to 99 %.
In the medical technology and sports industry, wearable devices, such as exoskeletons and running shoes, are developed to improve mobility and enhance performance. To ensure this, experimental studies are conducted in motion laboratories, where extensive measurements enable biomechanical movement and performance analysis. In a lab environment, however, natural movements can only be captured to a limited extent. Furthermore, recordings are cost- and time-intensive and an isolated consideration of all degrees of freedom in the design is practically impossible. Therefore, this thesis aims to bring human movement studies outside the lab in two ways. In order to enable studies in natural environments, novel methods for biomechanical gait analysis using Inertial Measurement Units (IMUs) are presented [Dor+19a; Dor+20]. In order to perform virtual studies for prediction of human-product interactions, a simulation framework is developed and evaluated for a use case in running shoe design [Dor+19a].
This cumulative thesis is built around three publications. In the first publication [Dor+19a], I showed with my co-authors that we can accurately estimate two-dimensional gait biomechanics from IMU data by solving a single optimal control problem. The advantage of this approach is that a dynamically consistent simulation of a musculoskeletal model is obtained, while sensor noise and drift are suppressed. In the second publication [Dor+20], we estimated two-dimensional gait biomechanics from IMU data using Convolutional Neural Networks (CNNs), which reduced computation time while achieving higher accuracy of kinetics compared to [Dor+19a]. We also showed that the generalization of CNNs can be improved for kinematics by augmenting the training dataset using optimal control simulation of musculoskeletal models. In the third publication [Dor+19b], my co-authors and I found that optimal control simulation of musculoskeletal models can predict differences in energy cost in running between two midsole materials, which were modeled with non-linear regression models using mechanical test data. This highlights the ability for studies in the virtual environment, which can further be used to answer “what if” questions in footwear design [Nit+21]. Additionally, the thesis contains an original literature review and a discussion of the contributed publications in the overarching context of this literature analysis.
In conclusion, this thesis shows that combining musculoskeletal optimal control with machine learning is a valuable approach for IMU-based biomechanical analysis of human gait and for predicting human responses. Follow-up work can extend these methods for three-dimensional movement studies [Nit+20], and further explore the combination of physics-based with machine learning models. This work contributes to the improvement of sports and medical products by enabling future analysis of human motion in natural environments and prediction of human-product interactions in a virtual testing environment.
X-ray computed tomography (CT) is a widely used imaging technique in both
medicine and industry. In recent years, robot-supported CT systems have been researched
and developed. These CT systems enable a variety of new possible scanning
trajectories. By tailoring these trajectories, one can reduce image artefacts and increase
the image quality. In industrial practice, robot-supported CT systems have
not yet been established. Among other things, this is due to a lack of automation.
The application-specific optimal views must be determined by a CT expert
who might need several tries before achieving the required task, which is costly and
time-consuming.
This work presents a holistic concept called Multipositional CT. Multipositional
CT aims to use X-ray projections from selected views to produce CT scans with
application-specific optimised image quality. The concept is based on two steps:
First, the X-rays that are not correctly or not reliably interpretable by the applied
reconstruction method are identified. The SART reconstruction method is extended
to reduce the influence of these X-rays on the CT result. Secondly, application-specific
optimal views are determined to ensure a sufficient amount of reliably interpretable
data for a complete reconstruction. An analysis and a review of view optimisation
methods in X-ray CT are presented, including the optimisation of circular trajectories,
the dynamic sampling of continuous trajectories and the optimisation of sets of
arbitrary views. Five key components for any CT view optimisation are identified:
the goal of the optimisation, the prior knowledge, the parameterisation, the figure of
merit and the optimisation procedure.
To ensure sufficient views with reliably interpretable data for a complete CT
reconstruction, figures of merits that rely on data completeness are developed. The
Tuy-Smith condition and the Nyquist-Shannon sampling theorem are combined to
generate a data completeness condition for arbitrary views that do not have to be on
a continuous curve. This condition is extended for scans on dense, metallic objects
by excluding those data that are identified as not reliably interpretable. Quantitative
quality measures are derived to assess the data completeness and data interpretability
of any CT scans.
Based on these quality measures, optimisation methods are developed for continuous
trajectories and for sets for arbitrary views. These methods optimise views
object-specifically for selectable object regions, but not for the detection of previously
defined surfaces or other so-called tasks. This ensures that even unexpected defects
or deviations in the regions of interest are digitised in the best possible way.
For industrial standard CT systems, methods for the optimisation of the object
position as well as methods for optimisation and fusion of multiple scans with different
object positions are developed. For more agile CT systems, e.g. C-arm CT systems or
twin robotic CT systems, methods for optimising sets of arbitrary views are created.
In experiments, both on simulated and real data, it is shown that the developed
metrics for data completeness correlate with established concepts of image quality.
The experiments demonstrate that the concept of Multipositional CT can be used
to reduce artefacts and thus extends the existing repertoire of artefact reduction
methods. With a detailed outlook on potential extensions and improvements to the
3D painting denotes the process of drawing digitally in a three-dimensional virtual space. In contrast to Virtual Reality (VR) or classic Augmented Reality (AR), projection mapping still represents a largely unknown way of visualizing the digital painting on the physical target surface itself via projection. In this doctoral thesis, we identify the main barriers that keep projection-mapping-based 3D painting, also called projection painting, from being applied as a practical tool in ordinary work environments. We present new projection painting techniques that, on the one hand, explicitly target the spontaneous nature of rapid prototyping workflows. Instead of expecting the user to arduously provide a digital model of their physical target object, the Scan&Paint system incrementally scans the object itself during the painting session and applies image-based rendering techniques to store the paint data and render the painting. On the other hand, the Proxy Painting system innovatively mixes digital and traditional painting methods to bridge the gap between these supposedly contradicting disciplines. A copy of the target object, the so-called proxy object, is painted by the user with traditional means and the paint is digitally recorded and processed for projection onto the real target surface. We further introduce a fast marker-based software synchronization method for both systems' projector and camera to accurately capture the projected imagery during scanning procedures at the devices' native frame rates without the need of additional hardware. The overarching goal of the presented techniques is a high level of practicability and accessibility to advance projection painting to a viable tool in 3D painting.
Sports analytics research has major impact on the development of innovative training methods and the broadcast of sports events. This dissertation provides algorithms for both kinematic analysis and performance interpretation based on unobtrusively obtained measurements from wearable sensors. Its main focus is set on the processing of 3D-orientation features and the exploration of their potential for sports analytics. The proposed algorithms are described and evaluated in five exemplary sports. In scuba diving, rowing and ski jumping, the 3D-orientation of the body/boat/skis is determined and further processed to analyze and visualize the motion behavior. In snowboarding and skateboarding, the board orientation is calculated and processed for motion visualization and machine learning. Board sport tricks are automatically detected and subsequently classified for trick category and type. The methods of this work were already partially applied for TV broadcast of international competitions (e.g., Olympics 2018). Additionally, they can support sports science research for establishing thorough investigations and innovative training methods.
System-Level Mapping, Analysis, and Management of Real-Time Applications in Many-Core Systems
(2021)
Heterogeneous many-core platforms are becoming the de facto standard architectures for embedded systems, fulfilling their high demand for computational power consequent to the rapid increase in the number and the workload of the applications in such systems. Contrarily to historically being statically one-time programmed and the mix of running applications being fixed, modern embedded systems are often subject to a highly dynamic environment in which the mix of concurrently executed applications, the availability of resources, and the non-functional requirements of each application may change at run time on-demand. This necessitates adaptive application mapping and system management schemes to effectively cope with the system dynamism. In recent years, applications with hard real-time requirements are being increasingly observed in emerging embedded systems in various time-critical areas, e.g., automotive, telecommunications, and avionics. In this context, novel design methodologies are required to enable an adaptive mapping and management of hard real-time applications on many-core platforms with verified worst-case timing guarantees.
Recently, so-called Hybrid Application Mapping (HAM) methodologies have been proposed that split the design process into an offline application mapping phase and an online system management phase and, thereby, can enable a dynamic mapping and management of hard real-time applications on many-core platforms. In HAM, for each application, a set of high-quality mapping alternatives with diverse resource requirements and verified timing guarantees are computed offline in a Design Space Exploration (DSE). These mappings are then used online by a Run-time Platform Manager (RPM) to launch the application on-demand by selecting a precomputed mapping that adheres to the current timing constraints of the application and the resource availability in the system. In line with this hybrid scheme, the work at hand proposes a collection of methodologies, analyses, and techniques that enable an automated mapping and adaptive management of hard real-time applications on heterogeneous many-core platforms with verified timing guarantees.
Foremost, a formal timing analysis technique is proposed which can derive worst-case timing guarantees for hard real-time applications in (heterogeneous) many-core systems. In light of this analysis, a novel mapping optimization approach is proposed in which the amount of resources allocated per mapping is fine-tuned according to the timing requirements of the application to obtain high-quality mappings with satisfactory timing guarantees and enhanced resource efficiency.
In many-core systems, hard real-time applications are particularly susceptible to built-in autonomous Dynamic Thermal Management (DTM) mechanisms which are responsible for preserving the thermal safety of the system. At the sight of thermal violations (overheating), DTM mechanisms suspend or decelerate the execution of the applications that are running in the thermally affected regions which may lead to the violation of their real-time constraints. To eliminate such fatal interferences, this work proposes a thermally composable HAM methodology that preserves the thermal safety of the system proactively at all times and, thereby, eliminates the exposure of real-time applications to DTM without jeopardizing the thermal integrity of the system.
The offline DSE in HAM typically delivers a large set of Pareto-optimal mappings which imposes a considerable management overhead on the RPM and may degrade its responsiveness. This can be particularly fatal for hard real-time applications due to their strict dependence on prompt reactions from the RPM, e.g., upon a change in their timing constraints. As a remedy, this work proposes an automatic mapping-set distillation methodology that distills a promising subset of the mappings for online use to alleviate the RPM’s overhead while retaining a diverse blend of quality- and resource-demand trade-offs to address various run-time scenarios.
For a seamless satisfaction of the timing constraints of hard real-time applications in many-core systems, on-the-fly adaptation of their mappings becomes inevitable in view of drastic changes in their timing constraints or the failure of resources they use, e.g., due to faults or overheating. To that end, this work proposes two mapping adaptation methodologies for hard real-time applications in many-core systems. In the first approach, adaptations are realized in the form of a reconfiguration between precomputed mappings, while the second approach empowers fine-grained mapping adaptations in the form of the migration of any subset of the application’s tasks to any desired locations. Both methodologies are supplied with formal timing analysis techniques and lightweight admission checks to verify the real-time conformity of each adaptation option online according to the current timing constraints of the application.
Standortbezogene Unterhaltung ist mittlerweile zu einem Grundbedürfnis geworden. Die erforderliche Genauigkeit und Zuverlässigkeit von Lokalisierungssystemen wächst nicht nur für intelligente Systeme wie selbstfahrende Fahrzeuge, Lieferdrohnen und mobile Geräte, sondern auch für alltägliche Fußgänger. Aufgrund der allgegenwärtigen Sensoren wie Kameras, GPS und Trägheitssensoren werden mit aufwendig handgefertigten Modellen und Algorithmen eine Vielzahl von Lokalisierungssystemen entwickelt. Um eine Einschränkung der freien Sicht und unterschiedliche Lichtverhältnisse von Kamerasystemen zu vermeiden werden typischerweise Funk- und Trägheitssensoren zur Lokalisierung verwendet. Unter idealen Laborbedingungen können diese Sensoren und Modelle, Positionen und Orientierungen langfristig genau abschätzen. In realen Umgebungen wirken sich jedoch viele Probleme wie ungenaue Systemmodellierung, unvollständige Sensormessungen, Rauschen und komplexe Umgebungsdynamiken auf die Genauigkeit und Zuverlässigkeit aus. Individuell betrachtet haben Funk- und Trägheitssensoren Schwierigkeiten: Funk lokalisiert aufgrund mehrerer Pfade durch statische oder dynamische Objekte entlang der Ausbreitungspfade zwischen Sender und Empfänger sehr ungenau. Im Gegensatz dazu akkumulieren Trägheitssensoren im Laufe der Zeit Entfernungs- und Orientierungsfehler und können keinen absoluten Bezug zur Weltkarte herstellen. Verfahren des Stands der Technik ergänzen beide Sensoren, um komplementäre Effekte zu verwenden, können jedoch die Schwierigkeiten nicht beheben. Darüber hinaus können sie mit einfachen Bewegungsmodellen wie konstanter Beschleunigung oder Geschwindigkeit keine stark nichtlinearen menschlichen Bewegungen beschreiben.
Das Hauptziel dieser Arbeit ist es daher, die Auswirkungen datengetriebener Methoden und verschiedener Sensordatenströme von lose platzierten Sensoren auf die Genauigkeit der Schätzung menschlicher Posen in hochdynamischen Situationen zu untersuchen. Die absolute Genauigkeit der erhaltenen Ergebnisse wird mit Filtermethoden nach dem Stand der Technik verglichen. Um die Probleme von Menschen entworfenen Lokalisierungsmodellen zu lösen, werden in dieser Arbeit maschinelle und tiefe Lernmethoden verwendet. Es werden Lernmethoden zur Positions‐, Geschwindigkeits- und Orientierungsschätzung sowie zur Rekonstruktion der Trajektorie unter Verwendung multimodaler Messungen von Funk- und Trägheitssensoren vorgestellt, um eine genaue und robuste Lokalisierung zu erreichen. Die Auswirkungen datengetriebener Verfahren entlang einer typischen Verarbeitungskette für die Lokalisierung mit Funk- und Trägheitssensoren werden untersucht. Die Verarbeitungskette ist lose gekoppelt in atomare Komponenten unterteilt, sodass jedes datengetriebene Verfahren problemlos ausgetauscht werden kann. Sequenzbasierte Lernmethoden werden entlang der Verarbeitungskette verwendet, um absolute Positionen aus Ankunftszeitstempeln von Funksignalen mit Mehrwegeausbreitung zu schätzen, ungerichtete Geschwindigkeitsvektoren von Trägheitssensoren zu schätzen, Bewegungsmuster zu klassifizieren, die die Ausrichtung der Trajektorie kalibrieren und um schließlich die einzelnen Komponenten zu einer Trajektorie zu fusionieren. Die vorgeschlagenen Methoden lernen, mit unterschiedlichem Bewegungsverhalten umzugehen und ermöglichen eine robuste und präzise Lokalisierung. Im Rahmen von Großstudien werden Mess- und Referenzdaten mit verschiedenen Bewegungsformen bei unterschiedlichen Geschwindigkeiten erfasst. Umfangreiche Experimente zeigen die Wirksamkeit und das Potenzial der vorgeschlagenen Methoden. Die datengetriebene, modulare Verarbeitungskette liefert genauere und robustere Schätzungen als bekannte Verfahren, auch bei dynamischen Bewegungen mit verrauschten Trägheitssensoren und Funkumgebungen mit Mehrwegeausbreitung.
Assistenzsystem für den wissensbasierten Aufbau konstruktionsbegleitender Finite-Elemente-Analysen
(2021)
Finite-Elemente-Simulationen sind in der heutigen Konstruktionspraxis unverzichtbar. Sie werden immer häufiger eingesetzt, um die Festigkeit der zunehmend komplexen Produkte zu überprüfen. Durch den frühestmöglichen Einsatz von Finite-Elemente-Analysen können hohe Kosten eingespart werden, da Konstruktionsfehler vermieden und weniger Prototypen gefertigt werden müssen. Für die Erstellung effizienter und zuverlässiger Simulationen ist jedoch umfangreiches Expertenwissen erforderlich und aus Kapazitätsgründen können erfahrene Simulationsingenieure nicht für jeden Auslegungsschritt hinzugezogen werden. Daher werden die Simulationen nicht früh genug eingesetzt oder müssen von Konstruktionsingenieuren erstellt werden, die häufig weniger Erfahrung in der Simulation haben. Die fehlende Erfahrung der Simulationsanwender kann daher zu ungeeigneten Finite-Elemente-Modellen und falschen Konstruktionsentscheidungen führen, die sehr kostenintensive und zeitaufwendige Iterationen in der Produktentwicklung zur Folge haben.
In dieser Arbeit wird ein wissensbasiertes Assistenzsystem entwickelt, das das notwendige Simulationswissen akquiriert und unerfahrenen Simulationsanwendern zur Verfügung stellt. Für den Aufbau der zugrundeliegenden Wissensbasis und die situative Unterstützung der Simulationsanwender wird ein neuartiger ontologiebasierter Ansatz vorgestellt. Die Innovation dieses Ansatzes liegt in der Adaption von KI-Methoden aus den Bereichen des Text Minings, Data Minings und Semantic Webs. Diese Methoden werden eingesetzt, um aus bestehenden, von erfahrenen Simulationsanwendern erstellten Simulationsmodellen und Textdokumenten das erforderliche Wissen zu extrahieren und gezielt abzurufen.
Data stream systems support queries on continuously arriving data. They provide similar query facilities like relational database systems. However, data stream systems continuously evaluate the queries on the arriving data and discard the data afterwards. Thus, it is possible to process high volumes of rapidly arriving data. Application examples are the monitoring of IT infrastructure or the processing of data from wireless sensor networks in wildland or animal surveillance scenarios. Compared to centralized data stream systems, distributed data stream systems can lower the resource demand, improve the performance and increase the lifetime of wireless sensor networks. This is particularly true if the data sources are already distributed and the hosts of the data sources take part in query processing.
In my dissertation, I investigate the interdependencies of logical query optimization and the assignment of operators to hosts for distributed data stream systems on heterogeneous hosts. In particular, I discuss the mathematical representation of selected optimization goals and constraints like resource limits for cost-based query optimization. I propose a technique to estimate whether logical query optimization steps may interfere with the subsequent assignment of operators to hosts. Well-known heuristic algorithms are adapted to the optimization problem of assigning operators to hosts. An evaluation compares the different algorithms. Moreover, I propose an algorithm for load balancing by multiple instantiation of operators.
Moderne Fahrzeuge stehen im Fokus einer umfassenden Elektrifizierung, welche eine signifikante Steigerung der im Fahrzeug verbauten Anzahl an elektrischen Motoren und Antriebe zur Folge hat. Hersteller elektrischer Motoren müssen diese Produktvielfalt beherrschen und zudem eine hohe Anpassungsfähigkeit hinsichtlich der Abwicklung ihrer Aufträge sowie eine schnelle Reaktionsfähigkeit aufweisen.
Vor diesem Hintergrund befasst sich diese Arbeit mit der Konzeption, der Entwicklung und der betrieblichen Einführung des wissensbasierten Assistenzsystems für elektrische Antriebe (engl. Drives-Development Assistance System, D-DAS). Um dies zu erreichen, werden zunächst die einzelnen Schritte der Entwicklung in einem adaptierten Elektromotor-Produktentstehungsprozess (aEMPEP) beschrieben und weiter unterteilt. Basierend auf diesem Prozess erfolgt die Konzeption des digitalen Assistenzsystems, das die Entwickler bei der Abarbeitung zeitaufwändiger, komplizierter oder fehleranfälliger Aufgaben unterstützen soll. Die Entwicklung, Implementierung und Einführung des Assistenzsystems in das fallgebende Unternehmen stellen ebenfalls den Gegenstand dieser Arbeit dar.
Die Untersuchungen ergeben, dass die prozessuale Aufbereitung die Grundlage für eine Digitalisierung, Automatisierung und wissensbasierte Unterstützung darstellt. Gleichermaßen konnte mit D-DAS die Frage beantwortet werden, wie ein wissensbasiertes Assistenzsystem zur Unterstützung der Produktentwicklung elektrischer Motoren konzipiert, entwickelt und eingeführt werden muss, damit es sich für den betrieblichen Einsatz in der Industrie eignet. Abschließend erfolgt die Evaluation von D-DAS und des adaptierten Elektromotor-Produktentstehungsprozesses mittels einer Fallstudie von vier Motorenprojekten, die in einer geeigneten Ergebnisqualität resultiert.
Since multicore platforms are becoming more common for real-time systems, multicore real-time scheduling algorithms become more relevant. For a more efficient use of the additional processing capacity of multiple cores, many of these algorithms allow tasks to migrate between cores, which improves the schedulability of task systems, but comes at the cost of additional migration overhead. One possible approach to reduce this overhead is the restriction of migration to predefined migration points that are known to be beneficial. For semipartitioned scheduling algorithms, an approach exists that identifies a set of potential migration points at which migrating tasks can be split statically, so that at run time, each job of the task migrates at this point. This has the disadvantage that it is impossible to avoid task migration, even if the actual run times of the current job would otherwise allow the job to finish on its current core.
This thesis presents a solution for this problem by introducing dynamic migration decisions. Instead of using a statically selected point, an algorithm for dynamic migration decisions tries to choose the latest possible migration point out of the predefined set, depending on the run-time behaviour of the current job. In order to use as much run-time information as possible, the presented algorithms use the concept of evaluation points, which are defined as the latest possible points at which migration decisions can be made. For each migrating job, an evaluation point is initially set and repeatedly recalculated, until migration decisions can no longer be delayed and a migration point is selected. From this approach, three algorithms are derived, which differ in the way evaluation points are defined. The algorithms presented in this thesis define evaluation points as points in code, points in execution time, and a combination of both, respectively.
All presented algorithms can be combined with any semipartitioned scheduling algorithm. It is shown that dynamic migration decisions do not cause any deadline misses in any task set, if the task set is schedulable with fixed migration points. An analysis of the additional required operations shows, that both the number of evaluation points, and the required effort for each evaluation point are at most logarithmic for all algorithms in the average case. Since the required effort decreases with increasing run times, the static assignment of split tasks to cores needs to consider only a constant overhead for each part of a split task.
All presented algorithms have been implemented on a Raspberry Pi v2 model B, based on a FreeRTOS port. Time measurements of this implementation indicate, that the overhead for dynamic migration decisions is unlikely to impact the schedulability of a given task system. While dynamic migration decisions make it possible to avoid migration if the current run times allow it, the measured response times show no significant adverse impact of the additional overhead of the presented algorithms.
With today’s technology, various non-invasive imaging methods provide detailed insight into the patient’s anatomy and support the physician during surgery. Depending on the signal type of each modality, the information collected is limited to narrow parts of the overall available information. A combination of several modalities makes it possible to increase the amount of information simultaneously made available to the physician. Especially a possible combination of the two most frequently used modalities, namely magnetic resonance imaging (MRI) and X-ray/computed tomography (CT) is of particular interest. Apart from the physical challenges, such a combination also poses a challenge to the acquisition scheme for a meaningful simultaneous acquisition with both modalities. This is especially true for the interventional environment, which implies additional requirements such as the real-time capability of the acquisition scheme. Such a combination, comes along with limitations for the signal recording schemes, up to the degree that an analytically correct solution can no longer be derived. Recently, there have been different attempts to overcome such limitations with machine learning. However, the solutions found are only partially comprehensible, and signal authenticity cannot be guaranteed. With the integration of prior knowledge about signal acquisition processes, such models can also be used in the medical field. In this thesis, we outline an acquisition scheme for a novel hybrid magnetic resonance (MR)/X-ray imaging system for the interventional environment and investigate the realization of parts of the scheme. Further, we investigate the concept of known operator learning, derive an implementation for operators in CT, and utilize the concept to enable the aforementioned hybrid MR/X-ray acquisition scheme. First, we investigate the possibilities and limitations for incorporating prior knowledge about the signal processing chain into machine learning algorithms. Using the universal approximation theorem (UAT), the benefits of mixing prior knowledge with neural networks are theoretically analyzed. Such a learning pipeline enriched with known operators allows the reduction of trainable parameters, and constrains the pipeline such that signal authenticity can be met, while the approximative power of deep learning (DL) can be utilized. In the course of the thesis, the concept is
implemented for the CT reconstruction and evaluated from a methodological and algorithmic point of view. The results show that CT operators can be integrated into neural networks allowing gradient flow through the whole pipeline. The experiments suggest that such mixed pipelines can be trained only with numerical data if designed specifically for the problem. Furthermore, an open-source software framework called PYRO-NN is developed to make these benefits accessible to a broader community. Second, we develop a novel hybrid MR/X-ray acquisition scheme for image-guided interventions. The scheme acknowledges the high contrast diversity of MRI and the benefits of fast, high-resolution X-ray imaging. The information provided by both modalities is captured in different domains, making a subsequent exploitation of the complementary signals difficult, and is identified as a major obstacle to fully benefit from the simultaneous hybrid acquisition. The proposed scheme results in multiple orthographic MR projections and, in one perspectively distorted X-ray projection per frame. In the course of the thesis, we first develop a novel tomographic conversion scheme to overcome the limitations of classical geometrical rebinning. The tomographic rebinning is derived and performed utilizing the known operator learning scheme to learn an efficient convolution-based algorithm for the conversion. The novel concept is first designed and comprehensively evaluated on the 1-D/2-D case against the baseline method. Subsequently, the tomographic rebinning concept is developed for the clinically relevant 2-D/3-D case. In the course of the experiments, we can show that the efficient convolution-based algorithm can be learned with purely numeric training data. The presented method overcomes the limitation of the baseline method and provides sharper and more distinct projection than the baseline method. Hence, the learned tomographic rebinning algorithm is promising for the proposed hybrid MR/X-ray acquisition scheme, and the underlying known operator learning concept encourages further integration of the acquisition and contrast parameters into the trainable pipeline. In general, the benefit of incorporating domain knowledge into neural networks is highly promising and is possible for various other tasks even beyond medical imaging.