Robust control for anaerobic digestion systems of Tequila vinasses under uncertainty: A Deep Deterministic Policy Gradient Algorithm - Mendiola-Rodriguez, Tannia A.; Ricardez-Sandoval, Luis A. (Digital Chemical Engineering, 2022)
•Reinforcement Learning is used as an optimal control scheme for Anaerobic Digestion.•Deep Deterministic Policy Gradient is employed as a learning strategy.•Disturbances and parametric uncertainty for Anaerobic Digestion are considered.•Single stage and two-stage model configurations are compared. The disposal of high concentrated Tequila vinasses is an environmental threat. An alternative to solve this problem is through anaerobic digestion processes to reduce organic matter while producing biogas. Anaerobic digestion is a complex system subject to external perturbations and parametric uncertainty; hence, control strategies are needed to guarantee operational efficiency under uncertainty. This study proposes an approach using an actor-critic model algorithm called Deep Deterministic Policy Gradient (DDPG) for a single-stage and a two-stage anaerobic digestion system to manage Tequila vinasses. To explore the feasibility of this algorithm, various scenarios expected during operation are investigated. To provide further insight, the algorithm's performance is compared between the two systems and compared to a conventional robust optimization formulation. In addition, a robust economic model predictive controller (EMPC) using the DDPG framework is tested to further illustrate their benefits. Results showed that the DDPG algorithm learned an optimal policy to minimize the organic matter content of tequila vinasses while producing biogas despite disturbances and parametric uncertainty. Also, the two-stage model exhibited better performance when compared to the single-stage model since significant improvements in methanogenic biomass and accumulated Chemical Oxygen Demand (COD) reduction were observed. While the DPPG algorithm showed an ability to learn optimal control policies under different scenarios, it still requires further improvements to realize their implementation for online large-scale applications.