Robust control for anaerobic digestion systems of Tequila vinasses under uncertainty: A Deep Deterministic Policy Gradient Algorithm

•Reinforcement Learning is used as an optimal control scheme for Anaerobic Digestion.•Deep Deterministic Policy Gradient is employed as a learning strategy.•Disturbances and parametric uncertainty for Anaerobic Digestion are considered.•Single stage and two-stage model configurations are compared. T...

Full description

Saved in:
Bibliographic Details
Published in:Digital Chemical Engineering Vol. 3; p. 100023
Main Authors: Mendiola-Rodriguez, Tannia A., Ricardez-Sandoval, Luis A.
Format: Journal Article
Language:English
Published: Elsevier Ltd 01.06.2022
Elsevier
Subjects:
Anaerobic Digestion
Deep Deterministic Policy Gradient
Process uncertainty
Reinforcement learning
Tequila vinasses
Tequila vinasses
Deep Deterministic Policy Gradient
Anaerobic Digestion
Process uncertainty
Reinforcement learning
ISSN:2772-5081, 2772-5081
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•Reinforcement Learning is used as an optimal control scheme for Anaerobic Digestion.•Deep Deterministic Policy Gradient is employed as a learning strategy.•Disturbances and parametric uncertainty for Anaerobic Digestion are considered.•Single stage and two-stage model configurations are compared. The disposal of high concentrated Tequila vinasses is an environmental threat. An alternative to solve this problem is through anaerobic digestion processes to reduce organic matter while producing biogas. Anaerobic digestion is a complex system subject to external perturbations and parametric uncertainty; hence, control strategies are needed to guarantee operational efficiency under uncertainty. This study proposes an approach using an actor-critic model algorithm called Deep Deterministic Policy Gradient (DDPG) for a single-stage and a two-stage anaerobic digestion system to manage Tequila vinasses. To explore the feasibility of this algorithm, various scenarios expected during operation are investigated. To provide further insight, the algorithm's performance is compared between the two systems and compared to a conventional robust optimization formulation. In addition, a robust economic model predictive controller (EMPC) using the DDPG framework is tested to further illustrate their benefits. Results showed that the DDPG algorithm learned an optimal policy to minimize the organic matter content of tequila vinasses while producing biogas despite disturbances and parametric uncertainty. Also, the two-stage model exhibited better performance when compared to the single-stage model since significant improvements in methanogenic biomass and accumulated Chemical Oxygen Demand (COD) reduction were observed. While the DPPG algorithm showed an ability to learn optimal control policies under different scenarios, it still requires further improvements to realize their implementation for online large-scale applications.
ISSN:2772-5081
2772-5081
DOI:10.1016/j.dche.2022.100023