Zone scheduling optimization of pumps in water distribution networks with deep reinforcement learning and knowledge-assisted learning

This article studies the pump scheduling optimization problem in water distribution networks (WDNs) through a novel algorithm that combines knowledge learning and deep reinforcement learning. The optimization problem is modeled as a Markov decision process by taking three objectives in pressure mana...

Full description

Saved in:
Bibliographic Details
Published in:Soft computing (Berlin, Germany) Vol. 25; no. 23; pp. 14757 - 14767
Main Authors: Xu, Jiahui, Wang, Hongyuan, Rao, Jun, Wang, Jingcheng
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.12.2021
Subjects:
Artificial Intelligence
Computational Intelligence
Control
Engineering
Mathematical Logic and Foundations
Mechatronics
Robotics
Soft Computing in Decision Making and in Modeling in Economics
Water distribution networks
Deep reinforcement learning
Knowledge-assisted learning
Optimal control
Pump schedule
ISSN:1432-7643, 1433-7479
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article studies the pump scheduling optimization problem in water distribution networks (WDNs) through a novel algorithm that combines knowledge learning and deep reinforcement learning. The optimization problem is modeled as a Markov decision process by taking three objectives in pressure management into consideration. Knowledge-assisted learning is incorporated into the reinforcement learning framework (KA-RL) to help evaluate the state value and guide the design of reward function, since the proposed KA-RL framework leverages the notion that historical data of WDNs could be utilized to produce optimal trajectories with respect to parametric variations. The knowledge-assisted proximal policy optimization (KA-PPO) algorithm, which only uses nodal pressure data, is proposed based on the KA-RL framework to address the arbitrary WDN topology and time-varying water demand. The effectiveness and applicability of the proposed algorithm are illustrated by virtue of the 22-node network with two pumps in a pump station. Empirical results demonstrate that KA-PPO works well in practice and compares favorably to Nelder–Mead method.
ISSN:1432-7643
1433-7479
DOI:10.1007/s00500-021-06177-3