Open Access

Many production lines require active control mechanisms, such as adaptive routing, worker reallocation, and rescheduling, to maintain optimal performance. However, designing these control systems is challenging for various reasons, and while reinforcement learning (RL) has shown promise in addressing these challenges, a standardized and general framework is still lacking. In this work, we introduce LineFlow, an extensible, open-source Python framework for simulating production lines of arbitrary complexity and training RL agents to control them. To demonstrate the capabilities and to validate the underlying theoretical assumptions of LineFlow, we formulate core subproblems of active line control in ways that facilitate mathematical analysis. For each problem, we provide optimal solutions for comparison. We benchmark state-of-the-art RL algorithms and show that the learned policies approach optimal performance in well-understood scenarios. However, for more complex, industrial-scale production lines, RL still faces significant challenges, highlighting the need for further research in areas such as reward shaping, curriculum learning, and hierarchical control.

This thesis presents ReLoop_RLive, a modular framework for vision-based active positioning research focused on the Sim2Real gap. The framework consists of two independent components. The first is a low-cost physical demonstrator built around the Sphero BOLT+ robot, accessed over HTTP. The second is a simulation backend whose physics and rendering components can be swapped independently. Two simulation backends of deliberately different fidelity are implemented, namely a high-fidelity SAPIEN engine with physically based rendering, and a lightweight Simple backend using a linear kinematic model and OpenCV rendering. The visual gap is quantified, where the SAPIEN-based backend achieves a lower score than Simple, confirming that its physically based renderer produces observations statistically closer to real camera images, though it still falls short of perfectly replicating reality. Physical gap analysis reveals significant displacement errors in both backends on short-duration trajectories. While both environments successfully train PPO agents under stationary, in-distribution conditions, and both agents generalize successfully between simulation backends, zero-shot transfer to the physical demonstrator reveals a clear asymmetry. The agent trained in the high-fidelity SAPIEN environment achieves a zero-shot success rate of 0.95 on the real robot, at the cost of a substantially increased mean episode length compared to simulation. The Simple-trained agent, by contrast, achieves only 0.4 on the real demonstrator, below the random-action baseline of 0.45, indicating that the policy provides no effective control on the physical system. Crucially, scaling the task difficulty to the originally intended random goal configuration presents an open problem, failing to converge across evaluated setups. This confirms that while high-fidelity rendering reduces the transfer gap, bridging the remaining visual and physical mismatches for dynamic active positioning remains an active challenge.

Manufacturing companies are increasingly confronted with critical challenges such as a shortage of skilled labor, rising production costs, and ever-stricter quality requirements. These challenges become particularly acute when defect types exhibit high visual variance, making consistent and accurate inspection difficult. Traditionally, visual inspection of high variance errors is performed manually by human operators—a process that is both costly and prone to errors. Consequently, there is a growing interest in replacing human inspection with AI-based visual quality control systems. However, the adoption of such systems is often hindered by limited access to training data, labor-intensive labeling processes, or the absence of real production data during early development stages. To address these challenges, this paper presents a methodology for training AI models using synthetically generated image data. The synthetic images are created using Physically Based Rendering, which enables precise control over rendering parameters and facilitates automated labeling. This approach allows for a systematic analysis of parameter importance and bypasses the need for large real training datasets. As a case study, the focus is on the inspection of laser welds in battery connectors for fully electric vehicles—a particularly demanding application due to the criticality of each weld. The results demonstrates the effectiveness of synthetic data in training robust AI models, thereby providing a scalable and efficient alternative to traditional data acquisition and labeling methods. The trained binary classifier reaches a precision of 0.94 with a recall of 0.98 solely trained on synthetic data and tested on real image data.

The response time of Artificial Neural Network (ANN) inference is critical in embedded systems processing sensor data close to the source. This is particularly important in applications such as predictive maintenance, which rely on timely state change predictions. This study enables estimation of model response times based on the underlying platform, highlighting the importance of benchmarking generic ANN applications on edge devices. We analyze the impact of network parameters, activation functions, and single- versus multi-threading on response times. Additionally, potential hardware-related influences, such as clock rate variances, are discussed. The results underline the complexity of task partitioning and scheduling strategies, stressing the need for precise parameter coordination to optimise performance across platforms. This study shows that cutting-edge frameworks do not necessarily perform the required operations automatically for all configurations, which may negatively impact performance. This paper further investigates the influence of network structure on model calibration, quantified using the Expected Calibration Error (ECE), and the limits of potential optimisation opportunities. It also examines the effects of model conversion to Tensorflow Lite (TFLite), highlighting the necessity of considering both performance and calibration when deploying models on embedded systems.

Process curves are multivariate finite time series data coming from manufacturing processes. This paper studies machine learning that detect drifts in process curve datasets. A theoretic framework to synthetically generate process curves in a controlled way is introduced in order to benchmark machine learning algorithms for process drift detection. An evaluation score, called the temporal area under the curve, is introduced, which allows to quantify how well machine learning models unveil curves belonging to drift segments. Finally, a benchmark study comparing popular machine learning approaches on synthetic data generated with the introduced framework is presented that shows that existing algorithms often struggle with datasets containing multiple drift segments.