RLiableDrives

Energy-efficient electric drives are a key technology for achieving Europe’s climate goals. Due to the high complexity of drive systems, operating strategies that do not achieve optimal energy efficiency are frequently used in practice. Methods from the field of artificial intelligence, particularly reinforcement learning (RL), enable automated learning of the control of sophisticated electric drives and are therefore a promising approach to this problem.
While our own preliminary work and that of other research groups demonstrate the effectiveness of these methods on the one hand, they also highlight challenges on the other, particularly regarding the desired training for end-use applications. Due to the methods and characteristic properties of the training, there is significant variation in the control performance of the learned controllers. Furthermore, it has been shown that training does not always reliably lead to a stable, usable operating strategy. Therefore, this project aims to investigate reliable and safe RL training. In this context, the term “reliable” implies a reduction in the variation of the control performance of the learned controllers trained with the same setup, so that controllers with 
optimal operating strategies are consistently trained. The term “safe” implies that training on the test bench is guaranteed to remain within the system’s physical limits (e.g., temperature, current limit, etc.) and without causing damage to the system.
To this end, ways to increase reliability are first being explored through simulation. RL training can be influenced by a variety of parameters. This project focuses on four areas of research: the effects of the RL algorithm itself, exploration, buffer sampling, and various environmental parameters, such as the reward function. 
Furthermore, strategies are being investigated that adhere to relevant limits across different operating ranges and dynamically adapt the control strategy while taking into account additional operating-point-dependent constraints. The insights gained from simulation are validated in test bench experiments and supplemented by empirical findings.
To this end, we are first exploring ways to improve reliability through simulation. RL training can be influenced by a variety of parameters. This project focuses on four key areas of research: the effects of the RL algorithm itself, exploration, buffer sampling, and various environmental parameters, such as the reward function. 
Furthermore, strategies are being investigated that adhere to relevant limits in various operating ranges and dynamically adapt the control strategy while taking into account additional operating-point-dependent specifications. The findings obtained in simulation are validated in test bench experiments and supplemented by empirical data. To this end, research is being conducted on hardware acceleration to enable real-time control using neural networks and learning on the test bench, as well as the advantageous implementation of the training process on the control platform.
The project could make RL-based control methods practical for electric drives and thus contribute significantly to increased efficiency and energy savings in industry. The planned open-source release of the software will enable broad use and further development of the research results.