Deep reinforcement learning based controller with dynamic feature extraction for an industrial claus process
•The DRL-based controller integrating with the process dynamics is developed.•The dynamic feature is extracted from the historical data using the Seq2seq network.•The controller was trained by interacting with the Seq2seq model.•The standard deviation of the control variable in the industrial Claus...
Uloženo v:
| Vydáno v: | Journal of the Taiwan Institute of Chemical Engineers Ročník 146; s. 104779 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Elsevier B.V
01.05.2023
|
| Témata: | |
| ISSN: | 1876-1070, 1876-1089 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | •The DRL-based controller integrating with the process dynamics is developed.•The dynamic feature is extracted from the historical data using the Seq2seq network.•The controller was trained by interacting with the Seq2seq model.•The standard deviation of the control variable in the industrial Claus process can be reduced by up to 55% using the proposed DRL-based controller.
The significant time delay between the manipulated and controlled variables introduces challenges in the task of system identification when implementing model predictive control (MPC) for an industrial process. Recently, deep reinforcement learning (DRL) with model-free characteristics has attracted considerable attention from the process control community. However, the model-free assumption in DRL is based on the property of the Markov decision process (MDP), in which all state variables must be observed. This assumption is not true for an industrial process.
In this study, the sequence-to-sequence (Seq2seq) network was employed to build a surrogate model based on the industrial Claus process data. Meanwhile, the hidden state output from the encoder of the Seq2seq network, which represents the dynamic feature of the process, connects to the DRL-based controller to compensate for the partial observabilities of a real process.
The results show that the standard deviation of the control variable, which refers to the H2S to SO2 concentration ratio in the tail gas, can be reduced by up to 55% using the proposed DRL-based controller comparing with the current control strategy.
[Display omitted] |
|---|---|
| AbstractList | •The DRL-based controller integrating with the process dynamics is developed.•The dynamic feature is extracted from the historical data using the Seq2seq network.•The controller was trained by interacting with the Seq2seq model.•The standard deviation of the control variable in the industrial Claus process can be reduced by up to 55% using the proposed DRL-based controller.
The significant time delay between the manipulated and controlled variables introduces challenges in the task of system identification when implementing model predictive control (MPC) for an industrial process. Recently, deep reinforcement learning (DRL) with model-free characteristics has attracted considerable attention from the process control community. However, the model-free assumption in DRL is based on the property of the Markov decision process (MDP), in which all state variables must be observed. This assumption is not true for an industrial process.
In this study, the sequence-to-sequence (Seq2seq) network was employed to build a surrogate model based on the industrial Claus process data. Meanwhile, the hidden state output from the encoder of the Seq2seq network, which represents the dynamic feature of the process, connects to the DRL-based controller to compensate for the partial observabilities of a real process.
The results show that the standard deviation of the control variable, which refers to the H2S to SO2 concentration ratio in the tail gas, can be reduced by up to 55% using the proposed DRL-based controller comparing with the current control strategy.
[Display omitted] |
| ArticleNumber | 104779 |
| Author | Liu, Jialin Tsai, Bing-Yen Chen, Ding-Sou |
| Author_xml | – sequence: 1 givenname: Jialin orcidid: 0000-0002-3180-6600 surname: Liu fullname: Liu, Jialin email: jialin@thu.edu.tw organization: Research Center for Smart Sustainable Circular Economy, Tunghai University, No. 1727, Sec.4, Taiwan Boulevard, Taichung, Taiwan – sequence: 2 givenname: Bing-Yen surname: Tsai fullname: Tsai, Bing-Yen organization: Department of Chemical and Materials Engineering, Tunghai University, No. 1727, Sec.4, Taiwan Boulevard, Taichung, Taiwan – sequence: 3 givenname: Ding-Sou surname: Chen fullname: Chen, Ding-Sou organization: Green Energy and System Integration Research and Development Department, China Steel Corporation, No. 1, Chung Kang Road, Hsiao Kang, Kaohsiung, Taiwan |
| BookMark | eNqFkMtOwzAQRS1UJErpF7DxD6TYThonCxaoPKVKbGBtOeMJOHKdynaB_j0JRSxYwGxmNNK50j2nZOJ7j4Scc7bgjJcX3aJLFnAhmMiHTyFlfUSmvJJlxllVT35uyU7IPMaOjSOZEMspcdeIWxrQ-rYPgBv0iTrUwVv_Qhsd0VDofQq9cxjou02v1Oy93ligLeq0C0jxIwUNyfaeDhlUe2q92cUUrHYUnN5Fug09YIxn5LjVLuL8e8_I8-3N0-o-Wz_ePayu1hnkLE8Zr3QjAHRZANNC1CIXQoMoi0IWYHTFDKBk0FYGRcNRNq3kS8O4zNuSVYj5jOSHXAh9jAFbtQ12o8NecaZGZ6pTX87U6EwdnA1U_YsCm_TYa-hn3T_s5YHFodabxaAiWPSAxgaEpExv_-Q_ASEDjds |
| CitedBy_id | crossref_primary_10_1016_j_jtice_2024_105864 crossref_primary_10_1016_j_jtice_2025_106119 crossref_primary_10_1016_j_jtice_2024_105407 crossref_primary_10_1016_j_supflu_2025_106542 crossref_primary_10_1016_j_jmsy_2025_07_002 crossref_primary_10_1016_j_jtice_2024_105430 crossref_primary_10_1007_s10489_024_05943_8 |
| Cites_doi | 10.1016/j.jprocont.2018.11.004 10.3115/v1/W14-4012 10.1016/j.compchemeng.2020.107016 10.1002/aic.17306 10.1162/neco.1997.9.8.1735 10.1016/j.compchemeng.2019.106649 10.1016/j.fuel.2013.05.070 10.1038/nature16961 10.1016/S0004-3702(98)00023-X 10.3115/v1/D14-1179 10.1016/j.compchemeng.2014.01.019 10.1016/j.jtice.2021.104200 10.1016/j.jtice.2021.04.062 10.1016/j.jtice.2022.104498 10.1038/nature14236 10.1021/acs.iecr.1c04984 10.1002/aic.14523 10.1109/TII.2019.2952429 10.1207/s15516709cog1402_1 10.1016/j.jtice.2022.104445 10.1016/j.compchemeng.2017.10.008 10.1016/j.jtice.2021.06.050 10.1002/aic.16689 10.1016/j.engappai.2018.07.003 10.1038/nature14539 10.1016/j.compchemeng.2022.107987 10.1016/j.jclepro.2021.125915 10.1016/j.compchemeng.2021.107280 10.1016/j.compchemeng.2020.107077 10.1016/j.compchemeng.2019.05.029 10.1021/acs.iecr.1c04668 10.1016/j.jprocont.2021.06.004 |
| ContentType | Journal Article |
| Copyright | 2023 Taiwan Institute of Chemical Engineers |
| Copyright_xml | – notice: 2023 Taiwan Institute of Chemical Engineers |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.jtice.2023.104779 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Economics Engineering |
| EISSN | 1876-1089 |
| ExternalDocumentID | 10_1016_j_jtice_2023_104779 S1876107023001086 |
| GroupedDBID | --K --M .DC .~1 0R~ 1B1 1~. 1~5 4.4 457 4G. 5VS 7-5 71M 8P~ 8RM AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAXUO ABFNM ABJNI ABMAC ABNUV ABXDB ABYKQ ACDAQ ACGFS ACRLP ADBBV ADEWK ADEZE ADMUD AEBSH AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHPOS AIEXJ AIKHN AITUG AJBFU AJOXV AKURH ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AXJTR BKOJK BLXMC DU5 EBS EFJIC EFLBG EJD ENUVR EP2 EP3 FDB FEDTE FIRID FNPLU FYGXN GBLVA HVGLF HZ~ J1W KOM M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 RIG ROL SDF SES SPC SPCBC SSG SSZ T5K ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD |
| ID | FETCH-LOGICAL-c303t-18ab2cca64c0a2292322ac264474cda80dce70cf8de2b1e7bf715d0173f608ee3 |
| ISICitedReferencesCount | 9 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000951950000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1876-1070 |
| IngestDate | Sat Nov 29 06:58:29 EST 2025 Tue Nov 18 20:49:11 EST 2025 Fri Feb 23 02:37:57 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Deep reinforcement learning industrial Claus process Actor-critic networks Sequence-to-sequence network Twin delayed deep deterministic policy gradient algorithm |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c303t-18ab2cca64c0a2292322ac264474cda80dce70cf8de2b1e7bf715d0173f608ee3 |
| ORCID | 0000-0002-3180-6600 |
| ParticipantIDs | crossref_primary_10_1016_j_jtice_2023_104779 crossref_citationtrail_10_1016_j_jtice_2023_104779 elsevier_sciencedirect_doi_10_1016_j_jtice_2023_104779 |
| PublicationCentury | 2000 |
| PublicationDate | May 2023 2023-05-00 |
| PublicationDateYYYYMMDD | 2023-05-01 |
| PublicationDate_xml | – month: 05 year: 2023 text: May 2023 |
| PublicationDecade | 2020 |
| PublicationTitle | Journal of the Taiwan Institute of Chemical Engineers |
| PublicationYear | 2023 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Mowbray, Smith, del Rio-Chanona, Zhang (bib0025) 2021; 67 Mokhatab, Poe, Speight (bib0041) 2006 Chou, Wu, Kang, Wong, Yao, Chuang, Jang, Ou (bib0024) 2020; 16 Elman (bib0027) 1990; 14 Cho K., Van Merriënboer B., Bahdanau D., Bengio Y. On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint 2014; arXiv:1409.1259. Fujimoto, Hoof, Meger (bib0035) 2018 Dogru, Wieczorek, Velswamy, Ibrahim, Huang (bib0018) 2021; 104 Mnih, Kavukcuoglu, Silver, Rusu, Veness, Bellemare, Ostrovski (bib0008) 2015; 518 Schulman J., Wolski F., Dhariwal P., Radford A., Klimov O. Proximal policy optimization algorithms. arXiv preprint 2017; arXiv:1707.06347v2. Byun, Kim, Lee (bib0016) 2022; 167 Manenti, Papasidero, Bozzano, Ranzi (bib0040) 2014; 66 Kaelbling, Littman, Cassandra (bib0026) 1998; 101 Katharopoulos A., Vyas A., Pappas N., Fleuret F. Transformers are RNNs: fast autoregressive transformers with linear attention. arXiv preprint 2020; arXiv:2006.16236v3. Lee, Shin, Realff (bib0002) 2018; 114 He, Shi, Tan, Song, Zhu (bib0004) 2021; 122 Haarnoja T., Zhou A., Abbeel P., Levine S. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint 2018; arXiv:1801.01290v2. LeCun, Bengio, Hinton (bib0003) 2015; 521 Shin, Badgwell, Liu, Lee (bib0010) 2019; 127 Oh, Adams, Vo, Gbadago, Lee, Oh (bib0021) 2021; 149 Kang, Mirzaei, Zhou (bib0012) 2022; 130 Sutton, Barto (bib0017) 2018 Qin (bib0001) 2014; 60 Zhou, Xu, He, Zhang (bib0006) 2022; 138 Adams, Oh, Kim, Lee, Oh (bib0023) 2021; 291 Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser L., Polosukhin I. Attention is all you need. arXiv preprint 2017; arXiv:1706.03762v5. Petsagkourakis, Sandoval, Bradford, Zhang, del Rio-Chanona (bib0014) 2020; 133 Razzaq, Li, Zhang (bib0038) 2013; 113 Zheng, Wu, Sun, Song, Zhou (bib0007) 2022; 138 Ma, Noreña-Caro, Adams, Brentzel, Romagnoli, Benton (bib0015) 2020; 142 Cho K., Van Merriënboer B., Gulcehre C., Bahdanau D., Bougares F., Schwenk H., Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint 2014; arXiv:1406.1078. Spielberg, Tulsyan, Lawrence, Loewen, Gopaluni (bib0011) 2019; 65 Silver, Huang, Maddison, Guez, Sifre, Van Den Driessche, Lanctot (bib0009) 2016; 529 Ma, Zhu, Benton, Romagnoli (bib0013) 2019; 75 Lillicrap T.P., Hunt J.J., Pritzel A. Heess N Erez T Tassa Y., Silver D., Wierstra D. Continuous control with deep reinforcement learning. arXiv preprint 2015; arXiv:1509.02971. Moral, Ortiz-Imedio, Ortiz, Gorri, Ortiz (bib0039) 2022; 61 Campos, El-Farra, Palazoglu (bib0019) 2022; 61 Powell, Machalek, Quah (bib0020) 2020; 143 Cheng, Huang, Pang, Zhang (bib0022) 2018; 74 Hochreiter, Schmidhuber (bib0028) 1997; 9 Silver, Lever, Heess, Degris, Wierstra, Riedmiller (bib0036) 2014 Yu, Liu, Liu, Wang (bib0005) 2022; 132 Elman (10.1016/j.jtice.2023.104779_bib0027) 1990; 14 Moral (10.1016/j.jtice.2023.104779_bib0039) 2022; 61 Chou (10.1016/j.jtice.2023.104779_bib0024) 2020; 16 Mnih (10.1016/j.jtice.2023.104779_bib0008) 2015; 518 Zheng (10.1016/j.jtice.2023.104779_bib0007) 2022; 138 Hochreiter (10.1016/j.jtice.2023.104779_bib0028) 1997; 9 He (10.1016/j.jtice.2023.104779_bib0004) 2021; 122 Dogru (10.1016/j.jtice.2023.104779_bib0018) 2021; 104 10.1016/j.jtice.2023.104779_bib0030 Silver (10.1016/j.jtice.2023.104779_bib0009) 2016; 529 Qin (10.1016/j.jtice.2023.104779_bib0001) 2014; 60 Razzaq (10.1016/j.jtice.2023.104779_bib0038) 2013; 113 Byun (10.1016/j.jtice.2023.104779_bib0016) 2022; 167 Oh (10.1016/j.jtice.2023.104779_bib0021) 2021; 149 Campos (10.1016/j.jtice.2023.104779_bib0019) 2022; 61 Sutton (10.1016/j.jtice.2023.104779_bib0017) 2018 Shin (10.1016/j.jtice.2023.104779_bib0010) 2019; 127 Mowbray (10.1016/j.jtice.2023.104779_bib0025) 2021; 67 Spielberg (10.1016/j.jtice.2023.104779_bib0011) 2019; 65 10.1016/j.jtice.2023.104779_bib0029 Zhou (10.1016/j.jtice.2023.104779_bib0006) 2022; 138 Mokhatab (10.1016/j.jtice.2023.104779_bib0041) 2006 Manenti (10.1016/j.jtice.2023.104779_bib0040) 2014; 66 Lee (10.1016/j.jtice.2023.104779_bib0002) 2018; 114 Ma (10.1016/j.jtice.2023.104779_bib0015) 2020; 142 Kaelbling (10.1016/j.jtice.2023.104779_bib0026) 1998; 101 Adams (10.1016/j.jtice.2023.104779_bib0023) 2021; 291 Powell (10.1016/j.jtice.2023.104779_bib0020) 2020; 143 LeCun (10.1016/j.jtice.2023.104779_bib0003) 2015; 521 Cheng (10.1016/j.jtice.2023.104779_bib0022) 2018; 74 Yu (10.1016/j.jtice.2023.104779_bib0005) 2022; 132 10.1016/j.jtice.2023.104779_bib0037 10.1016/j.jtice.2023.104779_bib0031 10.1016/j.jtice.2023.104779_bib0032 10.1016/j.jtice.2023.104779_bib0033 Kang (10.1016/j.jtice.2023.104779_bib0012) 2022; 130 10.1016/j.jtice.2023.104779_bib0034 Ma (10.1016/j.jtice.2023.104779_bib0013) 2019; 75 Silver (10.1016/j.jtice.2023.104779_bib0036) 2014 Fujimoto (10.1016/j.jtice.2023.104779_bib0035) 2018 Petsagkourakis (10.1016/j.jtice.2023.104779_bib0014) 2020; 133 |
| References_xml | – volume: 167 year: 2022 ident: bib0016 article-title: Multi-step lookahead Bayesian optimization with active learning using reinforcement learning and its application to data-driven batch-to-batch optimization publication-title: Comput Chem Eng – reference: Katharopoulos A., Vyas A., Pappas N., Fleuret F. Transformers are RNNs: fast autoregressive transformers with linear attention. arXiv preprint 2020; arXiv:2006.16236v3. – volume: 66 start-page: 244 year: 2014 end-page: 251 ident: bib0040 article-title: Model-based optimization of sulfur recovery units publication-title: Comp Chem Eng – volume: 75 start-page: 40 year: 2019 end-page: 47 ident: bib0013 article-title: Continuous control of a polymerization system with deep reinforcement learning publication-title: J Process Control – volume: 14 start-page: 179 year: 1990 end-page: 211 ident: bib0027 article-title: Finding structure in time publication-title: Cogn Sci – volume: 61 start-page: 6106 year: 2022 end-page: 6124 ident: bib0039 article-title: Hydrogen recovery from coke oven gas. Comparative analysis of technical alternatives publication-title: Ind Eng Chem Res – volume: 9 start-page: 1735 year: 1997 end-page: 1780 ident: bib0028 article-title: Long short-term memory publication-title: Neural Comput – reference: Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser L., Polosukhin I. Attention is all you need. arXiv preprint 2017; arXiv:1706.03762v5. – volume: 130 year: 2022 ident: bib0012 article-title: Robust control and training risk reduction for boiler level control using two-stage training deep deterministic policy gradient publication-title: J Taiwan Inst Chem Eng – volume: 143 year: 2020 ident: bib0020 article-title: Real-time optimization using reinforcement learning publication-title: Comp Chem Eng – reference: Lillicrap T.P., Hunt J.J., Pritzel A. Heess N Erez T Tassa Y., Silver D., Wierstra D. Continuous control with deep reinforcement learning. arXiv preprint 2015; arXiv:1509.02971. – reference: Cho K., Van Merriënboer B., Bahdanau D., Bengio Y. On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint 2014; arXiv:1409.1259. – volume: 529 start-page: 484 year: 2016 end-page: 489 ident: bib0009 article-title: Mastering the game of Go with deep neural networks and tree search publication-title: Nature – volume: 104 start-page: 86 year: 2021 end-page: 100 ident: bib0018 article-title: Online reinforcement learning for a continuous space system with experimental validation publication-title: J Process Control – reference: Haarnoja T., Zhou A., Abbeel P., Levine S. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint 2018; arXiv:1801.01290v2. – volume: 61 start-page: 8443 year: 2022 end-page: 8461 ident: bib0019 article-title: Soft actor-critic deep reinforcement learning with hybrid mixed-integer actions for demand responsive scheduling of energy systems publication-title: Ind Eng Chem Res – year: 2014 ident: bib0036 article-title: Deterministic policy gradient algorithms publication-title: Proceedings of the 31st international conference on machine learning – start-page: 626 year: 2006 ident: bib0041 article-title: Handbook of natural gas transmission and processing – volume: 138 year: 2022 ident: bib0007 article-title: Deep learning of complex process data for fault classification based on sparse probabilistic dynamic network publication-title: J Taiwan Inst Chem Eng – volume: 16 start-page: 2829 year: 2020 end-page: 2838 ident: bib0024 article-title: Physically consistent soft-sensor development using sequence-to-sequence neural networks publication-title: IEEE T Ind Inform – volume: 133 year: 2020 ident: bib0014 article-title: Reinforcement learning for batch bioprocess optimization publication-title: Comp Chem Eng – volume: 291 year: 2021 ident: bib0023 article-title: Deep reinforcement learning optimization framework for a power generation plant considering performance and environmental issues publication-title: J Clean Prod – volume: 74 start-page: 303 year: 2018 end-page: 311 ident: bib0022 article-title: ThermalNet: a deep reinforcement learning-based combustion optimization system for coal-fired boiler publication-title: Eng Appl Artif Intell – volume: 521 start-page: 436 year: 2015 end-page: 444 ident: bib0003 article-title: Deep learning publication-title: Nature – volume: 114 start-page: 111 year: 2018 end-page: 121 ident: bib0002 article-title: Machine learning: overview of the recent progresses and implications for the process systems engineering field publication-title: Comp Chem Eng – volume: 65 start-page: e16689 year: 2019 ident: bib0011 article-title: Toward self-driving processes: a deep reinforcement learning approach to control publication-title: AIChE J – volume: 138 year: 2022 ident: bib0006 article-title: Online abnormal interval detection and classification of industrial time series data based on multi-scale deep learning publication-title: J Taiwan Inst Chem Eng – volume: 127 start-page: 282 year: 2019 end-page: 294 ident: bib0010 article-title: Reinforcement learning – overview of recent progress and implications for process control publication-title: Comp Chem Eng – volume: 113 start-page: 287 year: 2013 end-page: 299 ident: bib0038 article-title: Coke oven gas: availability, properties, purification, and utilization in China publication-title: Fuel – volume: 122 start-page: 78 year: 2021 end-page: 84 ident: bib0004 article-title: Multiblock temporal convolution network-based temporal-correlated feature learning for fault diagnosis of multivariate processes publication-title: J Taiwan Inst Chem Eng – volume: 518 start-page: 529 year: 2015 end-page: 533 ident: bib0008 article-title: Human-level control through deep reinforcement learning publication-title: Nature – reference: Schulman J., Wolski F., Dhariwal P., Radford A., Klimov O. Proximal policy optimization algorithms. arXiv preprint 2017; arXiv:1707.06347v2. – volume: 149 year: 2021 ident: bib0021 article-title: Actor-critic reinforcement learning to estimate the optimal operating conditions of the hydrocracking process publication-title: Comp Chem Eng – year: 2018 ident: bib0017 article-title: Reinforcement learning: an introduction – volume: 67 start-page: e17306 year: 2021 ident: bib0025 article-title: Using process data to generate an optimal control policy via apprenticeship and reinforcement learning publication-title: AIChE J – reference: Cho K., Van Merriënboer B., Gulcehre C., Bahdanau D., Bougares F., Schwenk H., Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint 2014; arXiv:1406.1078. – volume: 101 start-page: 99 year: 1998 end-page: 134 ident: bib0026 article-title: Planning and acting in partially observable stochastic domains publication-title: Artif Intell – volume: 60 start-page: 3092 year: 2014 end-page: 3100 ident: bib0001 article-title: Process data analytics in the era of big data publication-title: AIChE J – volume: 142 year: 2020 ident: bib0015 article-title: Machine-learning-based simulation and fed-batch control of cyanobacterial-phycocyanin production in Plectonema by artificial neural network and deep reinforcement learning publication-title: Comp Chem Eng – year: 2018 ident: bib0035 article-title: Addressing function approximation error in actor-critic methods publication-title: Proceedings of the 35th international conference on machine learning – volume: 132 year: 2022 ident: bib0005 article-title: Supervised convolutional autoencoder-based fault-relevant feature learning for fault diagnosis in industrial processes publication-title: J Taiwan Inst Chem Eng – volume: 75 start-page: 40 year: 2019 ident: 10.1016/j.jtice.2023.104779_bib0013 article-title: Continuous control of a polymerization system with deep reinforcement learning publication-title: J Process Control doi: 10.1016/j.jprocont.2018.11.004 – ident: 10.1016/j.jtice.2023.104779_bib0032 doi: 10.3115/v1/W14-4012 – volume: 142 year: 2020 ident: 10.1016/j.jtice.2023.104779_bib0015 article-title: Machine-learning-based simulation and fed-batch control of cyanobacterial-phycocyanin production in Plectonema by artificial neural network and deep reinforcement learning publication-title: Comp Chem Eng doi: 10.1016/j.compchemeng.2020.107016 – volume: 67 start-page: e17306 year: 2021 ident: 10.1016/j.jtice.2023.104779_bib0025 article-title: Using process data to generate an optimal control policy via apprenticeship and reinforcement learning publication-title: AIChE J doi: 10.1002/aic.17306 – start-page: 626 year: 2006 ident: 10.1016/j.jtice.2023.104779_bib0041 – volume: 9 start-page: 1735 year: 1997 ident: 10.1016/j.jtice.2023.104779_bib0028 article-title: Long short-term memory publication-title: Neural Comput doi: 10.1162/neco.1997.9.8.1735 – volume: 133 year: 2020 ident: 10.1016/j.jtice.2023.104779_bib0014 article-title: Reinforcement learning for batch bioprocess optimization publication-title: Comp Chem Eng doi: 10.1016/j.compchemeng.2019.106649 – ident: 10.1016/j.jtice.2023.104779_bib0037 – volume: 113 start-page: 287 year: 2013 ident: 10.1016/j.jtice.2023.104779_bib0038 article-title: Coke oven gas: availability, properties, purification, and utilization in China publication-title: Fuel doi: 10.1016/j.fuel.2013.05.070 – volume: 529 start-page: 484 year: 2016 ident: 10.1016/j.jtice.2023.104779_bib0009 article-title: Mastering the game of Go with deep neural networks and tree search publication-title: Nature doi: 10.1038/nature16961 – volume: 101 start-page: 99 year: 1998 ident: 10.1016/j.jtice.2023.104779_bib0026 article-title: Planning and acting in partially observable stochastic domains publication-title: Artif Intell doi: 10.1016/S0004-3702(98)00023-X – ident: 10.1016/j.jtice.2023.104779_bib0029 doi: 10.3115/v1/D14-1179 – ident: 10.1016/j.jtice.2023.104779_bib0033 – ident: 10.1016/j.jtice.2023.104779_bib0031 – volume: 66 start-page: 244 year: 2014 ident: 10.1016/j.jtice.2023.104779_bib0040 article-title: Model-based optimization of sulfur recovery units publication-title: Comp Chem Eng doi: 10.1016/j.compchemeng.2014.01.019 – volume: 132 year: 2022 ident: 10.1016/j.jtice.2023.104779_bib0005 article-title: Supervised convolutional autoencoder-based fault-relevant feature learning for fault diagnosis in industrial processes publication-title: J Taiwan Inst Chem Eng doi: 10.1016/j.jtice.2021.104200 – volume: 122 start-page: 78 year: 2021 ident: 10.1016/j.jtice.2023.104779_bib0004 article-title: Multiblock temporal convolution network-based temporal-correlated feature learning for fault diagnosis of multivariate processes publication-title: J Taiwan Inst Chem Eng doi: 10.1016/j.jtice.2021.04.062 – volume: 138 year: 2022 ident: 10.1016/j.jtice.2023.104779_bib0007 article-title: Deep learning of complex process data for fault classification based on sparse probabilistic dynamic network publication-title: J Taiwan Inst Chem Eng doi: 10.1016/j.jtice.2022.104498 – volume: 518 start-page: 529 year: 2015 ident: 10.1016/j.jtice.2023.104779_bib0008 article-title: Human-level control through deep reinforcement learning publication-title: Nature doi: 10.1038/nature14236 – volume: 61 start-page: 8443 year: 2022 ident: 10.1016/j.jtice.2023.104779_bib0019 article-title: Soft actor-critic deep reinforcement learning with hybrid mixed-integer actions for demand responsive scheduling of energy systems publication-title: Ind Eng Chem Res doi: 10.1021/acs.iecr.1c04984 – volume: 60 start-page: 3092 year: 2014 ident: 10.1016/j.jtice.2023.104779_bib0001 article-title: Process data analytics in the era of big data publication-title: AIChE J doi: 10.1002/aic.14523 – volume: 16 start-page: 2829 year: 2020 ident: 10.1016/j.jtice.2023.104779_bib0024 article-title: Physically consistent soft-sensor development using sequence-to-sequence neural networks publication-title: IEEE T Ind Inform doi: 10.1109/TII.2019.2952429 – year: 2018 ident: 10.1016/j.jtice.2023.104779_bib0017 – volume: 14 start-page: 179 year: 1990 ident: 10.1016/j.jtice.2023.104779_bib0027 article-title: Finding structure in time publication-title: Cogn Sci doi: 10.1207/s15516709cog1402_1 – volume: 138 year: 2022 ident: 10.1016/j.jtice.2023.104779_bib0006 article-title: Online abnormal interval detection and classification of industrial time series data based on multi-scale deep learning publication-title: J Taiwan Inst Chem Eng doi: 10.1016/j.jtice.2022.104445 – volume: 114 start-page: 111 year: 2018 ident: 10.1016/j.jtice.2023.104779_bib0002 article-title: Machine learning: overview of the recent progresses and implications for the process systems engineering field publication-title: Comp Chem Eng doi: 10.1016/j.compchemeng.2017.10.008 – volume: 130 year: 2022 ident: 10.1016/j.jtice.2023.104779_bib0012 article-title: Robust control and training risk reduction for boiler level control using two-stage training deep deterministic policy gradient publication-title: J Taiwan Inst Chem Eng doi: 10.1016/j.jtice.2021.06.050 – volume: 65 start-page: e16689 year: 2019 ident: 10.1016/j.jtice.2023.104779_bib0011 article-title: Toward self-driving processes: a deep reinforcement learning approach to control publication-title: AIChE J doi: 10.1002/aic.16689 – volume: 74 start-page: 303 year: 2018 ident: 10.1016/j.jtice.2023.104779_bib0022 article-title: ThermalNet: a deep reinforcement learning-based combustion optimization system for coal-fired boiler publication-title: Eng Appl Artif Intell doi: 10.1016/j.engappai.2018.07.003 – year: 2018 ident: 10.1016/j.jtice.2023.104779_bib0035 article-title: Addressing function approximation error in actor-critic methods – volume: 521 start-page: 436 year: 2015 ident: 10.1016/j.jtice.2023.104779_bib0003 article-title: Deep learning publication-title: Nature doi: 10.1038/nature14539 – volume: 167 year: 2022 ident: 10.1016/j.jtice.2023.104779_bib0016 article-title: Multi-step lookahead Bayesian optimization with active learning using reinforcement learning and its application to data-driven batch-to-batch optimization publication-title: Comput Chem Eng doi: 10.1016/j.compchemeng.2022.107987 – ident: 10.1016/j.jtice.2023.104779_bib0034 – volume: 291 year: 2021 ident: 10.1016/j.jtice.2023.104779_bib0023 article-title: Deep reinforcement learning optimization framework for a power generation plant considering performance and environmental issues publication-title: J Clean Prod doi: 10.1016/j.jclepro.2021.125915 – volume: 149 year: 2021 ident: 10.1016/j.jtice.2023.104779_bib0021 article-title: Actor-critic reinforcement learning to estimate the optimal operating conditions of the hydrocracking process publication-title: Comp Chem Eng doi: 10.1016/j.compchemeng.2021.107280 – volume: 143 year: 2020 ident: 10.1016/j.jtice.2023.104779_bib0020 article-title: Real-time optimization using reinforcement learning publication-title: Comp Chem Eng doi: 10.1016/j.compchemeng.2020.107077 – ident: 10.1016/j.jtice.2023.104779_bib0030 – volume: 127 start-page: 282 year: 2019 ident: 10.1016/j.jtice.2023.104779_bib0010 article-title: Reinforcement learning – overview of recent progress and implications for process control publication-title: Comp Chem Eng doi: 10.1016/j.compchemeng.2019.05.029 – year: 2014 ident: 10.1016/j.jtice.2023.104779_bib0036 article-title: Deterministic policy gradient algorithms – volume: 61 start-page: 6106 year: 2022 ident: 10.1016/j.jtice.2023.104779_bib0039 article-title: Hydrogen recovery from coke oven gas. Comparative analysis of technical alternatives publication-title: Ind Eng Chem Res doi: 10.1021/acs.iecr.1c04668 – volume: 104 start-page: 86 year: 2021 ident: 10.1016/j.jtice.2023.104779_bib0018 article-title: Online reinforcement learning for a continuous space system with experimental validation publication-title: J Process Control doi: 10.1016/j.jprocont.2021.06.004 |
| SSID | ssj0000070225 |
| Score | 2.3684473 |
| Snippet | •The DRL-based controller integrating with the process dynamics is developed.•The dynamic feature is extracted from the historical data using the Seq2seq... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 104779 |
| SubjectTerms | Actor-critic networks Deep reinforcement learning industrial Claus process Sequence-to-sequence network Twin delayed deep deterministic policy gradient algorithm |
| Title | Deep reinforcement learning based controller with dynamic feature extraction for an industrial claus process |
| URI | https://dx.doi.org/10.1016/j.jtice.2023.104779 |
| Volume | 146 |
| WOSCitedRecordID | wos000951950000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: ScienceDirect Freedom Collection customDbUrl: eissn: 1876-1089 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000070225 issn: 1876-1070 databaseCode: AIEXJ dateStart: 20090101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3db9MwELfKhgQ8TDBADBjyA28hKHHbOHkc-9BAaEJakcpT5NgXlCnKqrQd-w_4tzl_JRmFCZB4iSIrtlPfL-fz9e5-hLwuYpFkTINXSBFOgMehiFUSKhGhuSwBd4XCkE3ws7N0Ps8-jUbffS7MVc2bJr2-zhb_VdTYhsLWqbN_Ie5uUGzAexQ6XlHseP0jwR8BLIIWTEVUaZx_nhria6D3LOXD02torRtWWVb6oART5TNAfd06CnEbYxlUPcGHrMV6GSxsfsFvTFttzM5E9U00N4MRuuoEvgpiZ9B_rNYGTzhD1YcGLy1X9jt89fBLn7N26DJKjnT7-eV66LlggzhB607bSKkxGhjVM-4Nlk3kLQzbLNdQp7at63JjC7DeiAv946Wug8rG-n9sbilrfqqtfa5H1pPhQSzSnFN3yDbj0wzV4_bB--P5h85dp8siMUPh272gr2FlogU3Zvu1nTOwXWYPyY6TDD2wYHlERtDskns-J325Sx4MylI-JrWGEL0BIeohRA2EaA8hqiFEHYSogxDtIURxDCoa2kOIGghRB6En5PPJ8ezwNHSsHKFEc2cVxqkoGH73yURGgjE8IDAmpLar-UQqkUZKAo9kmSpgRQy8KHk8Vaj4x2USpQDjp2SruWzgGaFlqXg0jWJIx2jYsijjoIOSJZ45YiYmao8wv4a5dCXrNXNKnfvYxIvcLHyuFz63C79H3nSdFrZiy-2PJ144uTM6rTGZI6Bu6_j8Xzu-IPf7r-El2Vq1a9gnd-XVqlq2rxzwfgB3GKvZ |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Deep+reinforcement+learning+based+controller+with+dynamic+feature+extraction+for+an+industrial+claus+process&rft.jtitle=Journal+of+the+Taiwan+Institute+of+Chemical+Engineers&rft.au=Liu%2C+Jialin&rft.au=Tsai%2C+Bing-Yen&rft.au=Chen%2C+Ding-Sou&rft.date=2023-05-01&rft.pub=Elsevier+B.V&rft.issn=1876-1070&rft.eissn=1876-1089&rft.volume=146&rft_id=info:doi/10.1016%2Fj.jtice.2023.104779&rft.externalDocID=S1876107023001086 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1876-1070&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1876-1070&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1876-1070&client=summon |