Control of multi-phase machines is a challenging topic due to the high number of controlled variables. Conventional control methods, such as field-oriented control (FOC), address this issue by introducing more control loops. This, however, increases the controller design complexity, while the tuning process can become cumbersome. To tackle the above, this paper proposes a deep deterministic policy gradient algorithm based controller that fulfills all the control objectives in one computational stage. More specifically, the proposed approach aims to learn a suitable current control policy for six-phase permanent magnet synchronous machines to simplify the commissioning of the drive system. In doing so, physical limitations of the drive system can be accounted for, while the compensation of imbalances between the two three-phase subsystems is rendered possible. After validating the training results in a controller-in-the-loop environment, test bench measurements are provided to demonstrate the effectiveness of the proposed controller. As shown, favorable steady-state and dynamic performance is achieved that is comparable to that of FOC. Therefore, as indicated by the presented results, reinforcement learning-based control approaches for multi-phase machines is a promising research area.
This paper presents a current control approach for permanent magnet synchronous machines (PMSMs) using the deep reinforcement learning algorithm deep deterministic policy gradient (DDPG). The proposed method is designed by examining different training setups regarding the reward function, the observation vector, and the actor neural network. In doing so, the impact of the different design factors on the steady-state and dynamic behavior of the system is assessed, thus facilitating the selection of the setup that results in the most favorable performance. Moreover, to provide the necessary insight into the controller design, the entire path from training the agent in simulation, through testing the control in a controller-in-the-loop (CIL) environment, to deployment on the test bench is described. Subsequently, experimental results are provided, which show the efficacy of the presented algorithm over a wide range of operating points. Finally, in an attempt to promote open science and expedite the use of deep reinforcement learning in power electronic systems, the trained agents, including the CIL model, are rendered openly available and accessible such that reproducibility of the presented approach is possible.
This paper presents two rapid control prototyping (RCP) use cases facilitated by the open-source platform UltraZohm. The openly available UltraZohm development frame-
work eases the transition from simulation to the test bench. The framework offers the integration of automatic code generation for field-programmable gate arrays (FPGAs), either by using Simulink models based on the HDL Coder, or by synthesizing C++ code into VHDL via the Vivado high-level synthesis tool. The first use case focuses on the implementation details of an on-chip controller-in-the-loop setup, where a permanent magnetsynchronous machine is emulated in the FPGA with a sampling frequency of 2 MHz. The second use case presents an efficient real-time implemen-tation of the sphere decoding algorithm employed to solve the long-horizon finite control set model predictive control problem for a three-level neutral point clamped inverter driving an induction machine. Experimental results based on a small-scale prototype confirm that the algorithm can be executed in real time on the FPGA, with an execution time of a few tens of microseconds. Both use cases highlight the benefits of using a high-performance RCP platform for research in power electronics and their control.
Machine learning algorithms are increasingly used in industrial applications for a multitude of use-cases. However, using them in control tasks is a challenge due to real-time requirements and limited resources. In this paper, an implementation scheme for real-time inference of multilayer perceptron (MLP) neural networks on FPGAs is proposed. Design constraints for using MLPs in reinforcement learning agents for motor control applications are derived and accounted for in the implementation. Two MLP architectures are evaluated on an FPGA, and the timing and resource-usage data are reported. The real-time capability of the implementation for motor control applications is investigated for standard control frequencies. It is shown by experimental validation that real-time interference with an area-efficient implementation for motor control applications is achievable. Therefore, the proposed implementation scheme can be applied to deep reinforcement learning controllers with hard real-time requirements.
