Cocktail: Learn a Better Neural Network Controller from Multiple Experts via Adaptive Mixing and Robust Distillation

Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising performance without requiring the development of complex physical models; however, their adoption is significantly hindered by the concerns o...

Full description

Saved in:
Bibliographic Details
Published in:2021 58th ACM/IEEE Design Automation Conference (DAC) pp. 397 - 402
Main Authors: Wang, Yixuan, Huang, Chao, Wang, Zhilu, Xu, Shichao, Wang, Zhaoran, Zhu, Qi
Format: Conference Proceeding
Language:English
Published: IEEE 05.12.2021
Subjects:
Adaptive systems
Energy measurement
Neural networks
Reinforcement learning
Robustness
Time measurement
Weight measurement
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising performance without requiring the development of complex physical models; however, their adoption is significantly hindered by the concerns on their safety, robustness, and efficiency. In this work, we propose COCKTAIL, a novel design framework that automatically learns a neural network based controller from multiple existing control methods (experts) that could be either model-based or neural network based. In particular, COCKTAIL first performs reinforcement learning to learn an optimal system-level adaptive mixing strategy that incorporates the underlying experts with dynamically-assigned weights, and then conducts a teacher-student distillation with probabilistic adversarial training and regularization to synthesize a student neural network controller with improved control robustness (measured by a safe control rate metric with respect to adversarial attacks or measurement noises), control energy efficiency, and verifiability (measured by the computation time for verification). Experiments on three non-linear systems demonstrate significant advantages of our approach on these properties over various baseline methods.
AbstractList Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising performance without requiring the development of complex physical models; however, their adoption is significantly hindered by the concerns on their safety, robustness, and efficiency. In this work, we propose COCKTAIL, a novel design framework that automatically learns a neural network based controller from multiple existing control methods (experts) that could be either model-based or neural network based. In particular, COCKTAIL first performs reinforcement learning to learn an optimal system-level adaptive mixing strategy that incorporates the underlying experts with dynamically-assigned weights, and then conducts a teacher-student distillation with probabilistic adversarial training and regularization to synthesize a student neural network controller with improved control robustness (measured by a safe control rate metric with respect to adversarial attacks or measurement noises), control energy efficiency, and verifiability (measured by the computation time for verification). Experiments on three non-linear systems demonstrate significant advantages of our approach on these properties over various baseline methods.
Author Huang, Chao
Wang, Zhaoran
Wang, Yixuan
Wang, Zhilu
Xu, Shichao
Zhu, Qi
Author_xml – sequence: 1
  givenname: Yixuan
  surname: Wang
  fullname: Wang, Yixuan
  email: yixuanwang2024@u.northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 2
  givenname: Chao
  surname: Huang
  fullname: Huang, Chao
  email: chao.huang@northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 3
  givenname: Zhilu
  surname: Wang
  fullname: Wang, Zhilu
  email: zhilu.wang@u.northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 4
  givenname: Shichao
  surname: Xu
  fullname: Xu, Shichao
  email: shichaoxu2023@u.northwestern.edu
  organization: Northwestern University,Evanston,IL
– sequence: 5
  givenname: Zhaoran
  surname: Wang
  fullname: Wang, Zhaoran
  email: zhaoranwang@gmail.com
  organization: Northwestern University,Evanston,IL
– sequence: 6
  givenname: Qi
  surname: Zhu
  fullname: Zhu, Qi
  email: qzhu@northwestern.edu
  organization: Northwestern University,Evanston,IL
BookMark eNotkMtKAzEYRiMoqLVPIEJeoDX_5Dru6rReoFWQ7ksy84-EppMhk9b69hbs5pzFgW_x3ZLLLnZIyAOwKQArH-ezCgzTYlqwAqalNAqEuSDjUhtQSgpeaMGuyXgYvGOKSSNOvCG5ivU2Wx-e6BJt6qilz5gzJvqB-2TDSfknpi2tYpdTDOFU2hR3dLUP2fcB6eLYY8oDPXhLZ43tsz8gXfmj776p7Rr6Fd1-yHTuh-xDsNnH7o5ctTYMOD57RNYvi3X1Nll-vr5Xs-XEFkbnCQB3shQaGimdEYIJZXVdSAPONRYQWt5iy6FtlKgVSJBlyYoaaseE5shH5P5_1iPipk9-Z9Pv5nwN_wM6vV0N
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/DAC18074.2021.9586148
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9781665432740
1665432748
EndPage 402
ExternalDocumentID 9586148
Genre orig-research
GroupedDBID 6IE
6IH
ACM
ALMA_UNASSIGNED_HOLDINGS
CBEJK
RIE
RIO
ID FETCH-LOGICAL-a287t-113b59471d55b844046a7c2581bbda1e1f3fef31fd64c615159902c1cb0473e3
IEDL.DBID RIE
ISICitedReferencesCount 4
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000766079700067&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:28:30 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a287t-113b59471d55b844046a7c2581bbda1e1f3fef31fd64c615159902c1cb0473e3
PageCount 6
ParticipantIDs ieee_primary_9586148
PublicationCentury 2000
PublicationDate 2021-Dec.-5
PublicationDateYYYYMMDD 2021-12-05
PublicationDate_xml – month: 12
  year: 2021
  text: 2021-Dec.-5
  day: 05
PublicationDecade 2020
PublicationTitle 2021 58th ACM/IEEE Design Automation Conference (DAC)
PublicationTitleAbbrev DAC
PublicationYear 2021
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssib060584060
Score 2.2328107
Snippet Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising...
SourceID ieee
SourceType Publisher
StartPage 397
SubjectTerms Adaptive systems
Energy measurement
Neural networks
Reinforcement learning
Robustness
Time measurement
Weight measurement
Title Cocktail: Learn a Better Neural Network Controller from Multiple Experts via Adaptive Mixing and Robust Distillation
URI https://ieeexplore.ieee.org/document/9586148
WOSCitedRecordID wos000766079700067&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NawIxEA0qPfTUFi39Zg49dnXjbjab3uxa6aGKFA_eJB-zIMgqukp_fpPs1lLopaeEhBDIhJlM8t4LIY-55IyjzU5SylUQMyMDkTAdhBJTo3KtQvQiru98MknnczFtkKcjFwYRPfgMu67q3_LNWu_dVVlPsNTpVjZJk_Ok4mp97x33umdjU1iTdGgoesNBRp3Ui00C-7Rbj_31iYqPIaOz_81-Tjo_ZDyYHsPMBWlg0SZlZv2YA38-g1dIBQkvnpgDTm1Drmzh4d2QVVD0le1xTBIY1wBC8BrH5Q4OSwkDIzfO7cF4-WnnAFkY-Fir_a6EofMBqwow1yGz0essewvqDxQCaROhMqA0UkzY8GMYU6lTAkwk131mj6rKSIo0j3LMI5qbJNbV0UaEfU2thWIeYXRJWsW6wCsCKuUKE2oioXVMOaY04Yol1ssKbWgYXZO2W7DFppLIWNRrdfN38y05dTbxqBB2R1rldo_35EQfyuVu--Dt-gUhN6Vp
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Na8IwGH5xbrCdtqFj38thx1Ub2zTNbq5OHFOR4cGb5KsglCpaZT9_Sdo5BrvslJJSAnnD-9E8z_MCPKacEqpNdRJjKryQKO6xiEjP5zpWIpXC107EdUjH43g2Y5MaPO25MFprBz7TLfvo7vLVUm7tr7I2I7HVrTyAQ9s5q2JrfZ8ee79nopNf0XSwz9q9boKt2IspAzu4VX39q42KiyL90_-tfwbNHzoemuwDzTnUdN6AIjGezMI_n5HTSEUcvThqDrJ6GzwzgwN4o6QEo2fmjeWSoFEFIURO5bjYoN2Co67iK-v40GjxadZAPFfoYym2mwL1rBfISshcE6b912ky8KoWCh43pVDhYRwIwkwAUoSI2GoBRpzKDjHJqlAca5wGqU4DnKoolGVyw_yOxMZGIQ10cAH1fJnrS0AipkJHWAVMyhBTHeOIChIZP8ukwn5wBQ27YfNVKZIxr_bq-u_pBzgeTEfD-fBt_H4DJ9Y-DiNCbqFerLf6Do7krlhs1vfOxl_tbqiy
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2021+58th+ACM%2FIEEE+Design+Automation+Conference+%28DAC%29&rft.atitle=Cocktail%3A+Learn+a+Better+Neural+Network+Controller+from+Multiple+Experts+via+Adaptive+Mixing+and+Robust+Distillation&rft.au=Wang%2C+Yixuan&rft.au=Huang%2C+Chao&rft.au=Wang%2C+Zhilu&rft.au=Xu%2C+Shichao&rft.date=2021-12-05&rft.pub=IEEE&rft.spage=397&rft.epage=402&rft_id=info:doi/10.1109%2FDAC18074.2021.9586148&rft.externalDocID=9586148