Reinforcement learning to maximise wind turbine energy generation

We propose a reinforcement learning strategy to control wind turbine energy generation by actively changing the rotor speed, the rotor yaw angle and the blade pitch angle. A double deep Q-learning with a prioritized experience replay agent is coupled with a blade element momentum model and is trained to allow control for changing winds. The agent is trained to decide the best control (speed, yaw, pitch) for simple steady winds and is subsequently challenged with real dynamic turbulent winds, showing good performance. The double deep Q- learning is compared with a classic value iteration reinforcement learning control and both strategies outperform a classic PID control in all environments. Furthermore, the reinforcement learning approach is well suited to changing environments including turbulent/gusty winds, showing great adaptability. Finally, we compare all control strategies with real winds and compute the annual energy production. In this case, the double deep Q-learning algorithm also outperforms classic methodologies.

Unsupervised machine-learning shock-capturing technique for high-order solvers

We present a novel unsupervised machine learning shock capturing algorithm based on Gaussian Mixture Models (GMMs). The proposed GMM sensor demonstrates remarkable accuracy in detecting shocks and is robust across diverse test cases without the need for parameter tuning. We compare the GMM-based sensor with state-of-the-art alternatives. All methods are integrated into a high-order compressible discontinuous Galerkin solver where artificial viscosity can be modulated to capture shocks. Supersonic test cases, including high Reynolds numbers, showcase the sensor's performance, demonstrating the same effectiveness as fine-tuned state-of-the-art sensors. %The nodal DG aproach allows for potential applications in sub-cell flux-differencing formulations, supersonic feature detection, and mesh refinement. The adaptive nature and ability to function without extensive training datasets make this GMM-based sensor suitable for complex geometries and varied flow configurations. Our study reveals the potential of unsupervised machine learning methods, exemplified by the GMM sensor, to improve the robustness and efficiency of advanced CFD codes.

A reinforcement learning strategy for p-adaptation in high order solvers

Reinforcement learning (RL) has emerged as a promising approach to automating decision processes. This paper explores the application of RL techniques to optimise the polynomial order in the computational mesh when using high-order solvers. Mesh adaptation plays a crucial role in improving the efficiency of numerical simulations by improving accuracy while reducing the cost. Here, actor-critic RL models based on Proximal Policy Optimization offer a data-driven approach for agents to learn optimal mesh modifications based on evolving conditions. The paper provides a strategy for p-adaptation in high-order solvers and includes insights into the main aspects of RL-based mesh adaptation, including the formulation of appropriate reward structures and the interaction between the RL agent and the simulation environment. We discuss the impact of RL-based mesh p-adaptation on computational efficiency and accuracy. We test the RL p-adaptation strategy on a 1D inviscid Burgers' equation to demonstrate the effectiveness of the strategy. The RL strategy reduces the computational cost and improves accuracy over uniform adaptation, while minimising human intervention.