Cocktail: Learn a Better Neural Network Controller from Multiple Experts via Adaptive Mixing and Robust Distillation

Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising performance without requiring the development of complex physical models; however, their adoption is significantly hindered by the concerns o...

Full description

Saved in:
Bibliographic Details
Published in:2021 58th ACM/IEEE Design Automation Conference (DAC) pp. 397 - 402
Main Authors: Wang, Yixuan, Huang, Chao, Wang, Zhilu, Xu, Shichao, Wang, Zhaoran, Zhu, Qi
Format: Conference Proceeding
Language:English
Published: IEEE 05.12.2021
Subjects:
Adaptive systems
Energy measurement
Neural networks
Reinforcement learning
Robustness
Time measurement
Weight measurement
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising performance without requiring the development of complex physical models; however, their adoption is significantly hindered by the concerns on their safety, robustness, and efficiency. In this work, we propose COCKTAIL, a novel design framework that automatically learns a neural network based controller from multiple existing control methods (experts) that could be either model-based or neural network based. In particular, COCKTAIL first performs reinforcement learning to learn an optimal system-level adaptive mixing strategy that incorporates the underlying experts with dynamically-assigned weights, and then conducts a teacher-student distillation with probabilistic adversarial training and regularization to synthesize a student neural network controller with improved control robustness (measured by a safe control rate metric with respect to adversarial attacks or measurement noises), control energy efficiency, and verifiability (measured by the computation time for verification). Experiments on three non-linear systems demonstrate significant advantages of our approach on these properties over various baseline methods.
AbstractList Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising performance without requiring the development of complex physical models; however, their adoption is significantly hindered by the concerns on their safety, robustness, and efficiency. In this work, we propose COCKTAIL, a novel design framework that automatically learns a neural network based controller from multiple existing control methods (experts) that could be either model-based or neural network based. In particular, COCKTAIL first performs reinforcement learning to learn an optimal system-level adaptive mixing strategy that incorporates the underlying experts with dynamically-assigned weights, and then conducts a teacher-student distillation with probabilistic adversarial training and regularization to synthesize a student neural network controller with improved control robustness (measured by a safe control rate metric with respect to adversarial attacks or measurement noises), control energy efficiency, and verifiability (measured by the computation time for verification). Experiments on three non-linear systems demonstrate significant advantages of our approach on these properties over various baseline methods.
Author Huang, Chao
Wang, Zhaoran
Wang, Yixuan
Wang, Zhilu
Xu, Shichao
Zhu, Qi
Author_xml – sequence: 1
  givenname: Yixuan
  surname: Wang
  fullname: Wang, Yixuan
  email: yixuanwang2024@u.northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 2
  givenname: Chao
  surname: Huang
  fullname: Huang, Chao
  email: chao.huang@northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 3
  givenname: Zhilu
  surname: Wang
  fullname: Wang, Zhilu
  email: zhilu.wang@u.northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 4
  givenname: Shichao
  surname: Xu
  fullname: Xu, Shichao
  email: shichaoxu2023@u.northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 5
  givenname: Zhaoran
  surname: Wang
  fullname: Wang, Zhaoran
  email: zhaoranwang@gmail.com
  organization: Northwestern University,Evanston,IL
– sequence: 6
  givenname: Qi
  surname: Zhu
  fullname: Zhu, Qi
  email: qzhu@northwestern.edu
  organization: Northwestern University,Evanston,IL
BookMark eNotkMtKAzEYRiMoqLVPIEJeoDX_5Dru6rReoFWQ7ksy84-EppMhk9b69hbs5pzFgW_x3ZLLLnZIyAOwKQArH-ezCgzTYlqwAqalNAqEuSDjUhtQSgpeaMGuyXgYvGOKSSNOvCG5ivU2Wx-e6BJt6qilz5gzJvqB-2TDSfknpi2tYpdTDOFU2hR3dLUP2fcB6eLYY8oDPXhLZ43tsz8gXfmj776p7Rr6Fd1-yHTuh-xDsNnH7o5ctTYMOD57RNYvi3X1Nll-vr5Xs-XEFkbnCQB3shQaGimdEYIJZXVdSAPONRYQWt5iy6FtlKgVSJBlyYoaaseE5shH5P5_1iPipk9-Z9Pv5nwN_wM6vV0N
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/DAC18074.2021.9586148
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE/IET Electronic Library (IEL) (UW System Shared)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9781665432740
1665432748
EndPage 402
ExternalDocumentID 9586148
Genre orig-research
GroupedDBID 6IE
6IH
ACM
ALMA_UNASSIGNED_HOLDINGS
CBEJK
RIE
RIO
ID FETCH-LOGICAL-a287t-113b59471d55b844046a7c2581bbda1e1f3fef31fd64c615159902c1cb0473e3
IEDL.DBID RIE
ISICitedReferencesCount 4
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000766079700067&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:28:30 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a287t-113b59471d55b844046a7c2581bbda1e1f3fef31fd64c615159902c1cb0473e3
PageCount 6
ParticipantIDs ieee_primary_9586148
PublicationCentury 2000
PublicationDate 2021-Dec.-5
PublicationDateYYYYMMDD 2021-12-05
PublicationDate_xml – month: 12
  year: 2021
  text: 2021-Dec.-5
  day: 05
PublicationDecade 2020
PublicationTitle 2021 58th ACM/IEEE Design Automation Conference (DAC)
PublicationTitleAbbrev DAC
PublicationYear 2021
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssib060584060
Score 2.2328107
Snippet Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising...
SourceID ieee
SourceType Publisher
StartPage 397
SubjectTerms Adaptive systems
Energy measurement
Neural networks
Reinforcement learning
Robustness
Time measurement
Weight measurement
Title Cocktail: Learn a Better Neural Network Controller from Multiple Experts via Adaptive Mixing and Robust Distillation
URI https://ieeexplore.ieee.org/document/9586148
WOSCitedRecordID wos000766079700067&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3Na8IwFA9OdthpGzr2zTvsuGqTtmm6m6uTXRQZHrxJvgqCVNEq-_OXl3aOwS47NbSUQBLeR97v93uEPDmvzLmIaSBDpYNYuZxVKoaBXOHOhzAs5dI3m0gnEzGfZ9MWeT5yYay1Hnxmezj0tXyz1nu8KutniUDdyhNykqa85mp9nx2s7jnfFDYkHRpm_eEgpyj14pJARnvNv7-aqHgfMjr_3-wXpPtDxoPp0c1ckpYtO6TKnR1D8OcLeIVUkPDqiTmAahty5R4e3g15DUVfuS_IJIFxAyAEr3Fc7eCwlDAwcoNmD8bLTzcHyNLAx1rtdxUM0QasasBcl8xGb7P8PWgaKATSJUJVQGmkksy5H5MkSqASIJepZokLVZWR1NIiKmwR0cLwWNehTRYyTbUK4zSy0RVpl-vSXhNgqeJGmohpF0AxWQjKmYws1kg51VbckA4u2GJTS2QsmrW6_fv1HTnDPfGokOSetKvt3j6QU32olrvto9_XL5nHpFo
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Na8IwGA7ODbbTNnTse-9hx1WbtE3T3VydOKYiw4M3yVdBkCpaZT9_Sdo5Brvs1NBSAkl4P_I-z_Mi9Gi8MqUsxB73hfRCYXJWLogN5DJzPpgiMeWu2UQ8GrHpNBnX0NOeC6O1duAz3bJDV8tXS7m1V2XtJGJWt_IAHUZhSPySrfV9emx9z3gnv6LpYD9pdzsptmIvJg0kuFX9_auNivMivdP_zX-Gmj90PBjvHc05qum8gYrUWDIL_3wGp5EKHF4cNQes3gZfmIcDeENagtEX5ovlksCwghCCUzkuNrCbc-govrKGD4bzTzMH8FzBx1JsNwV0rRVYlJC5Jpr0Xidp36taKHjcpEKFh3EgosQ4IBVFglktQMpjSSITrArFscZZkOkswJmioSyDm8QnEkvhh3GggwtUz5e5vkRAYkEVVwGRJoQiPGOYEh5oWyWlWGp2hRp2wWarUiRjVq3V9d-vH9BxfzIczAZvo_cbdGL3x2FEoltUL9ZbfYeO5K6Yb9b3bo-_AFMup6E
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2021+58th+ACM%2FIEEE+Design+Automation+Conference+%28DAC%29&rft.atitle=Cocktail%3A+Learn+a+Better+Neural+Network+Controller+from+Multiple+Experts+via+Adaptive+Mixing+and+Robust+Distillation&rft.au=Wang%2C+Yixuan&rft.au=Huang%2C+Chao&rft.au=Wang%2C+Zhilu&rft.au=Xu%2C+Shichao&rft.date=2021-12-05&rft.pub=IEEE&rft.spage=397&rft.epage=402&rft_id=info:doi/10.1109%2FDAC18074.2021.9586148&rft.externalDocID=9586148