Using Behavioural Programming with Solver, Context, and Deep Reinforcement Learning for Playing a Simplified RoboCup-Type Game

We describe four scenario-based implementations of controllers for a player in a simplified RoboCup-type game. All four implementations are based on the behavioural programming (BP) approach. We first describe a simple controller for the player using the state-of-the-art BPjs tool and then show how...

Full description

Saved in:
Bibliographic Details
Published in:2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) pp. 243 - 251
Main Authors: Elyasaf, Achiya, Sadon, Aviran, Weiss, Gera, Yaacov, Tom
Format: Conference Proceeding
Language:English
Published: IEEE 01.09.2019
Subjects:
Behavioral Programming
BPjs
Context modeling
Context Oriented Modelling
Couplings
Deep reinforcement learning
DRL
Games
Intelligent agents
Manuals
Model driven engineering
Programming
Robots
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract We describe four scenario-based implementations of controllers for a player in a simplified RoboCup-type game. All four implementations are based on the behavioural programming (BP) approach. We first describe a simple controller for the player using the state-of-the-art BPjs tool and then show how it can be extended in various ways. The first extension is based on a version of BP where the Z3 SMT solver is used to provide mechanisms for richer composition of modules within the BP model. This allows for modules with higher cohesion and lower coupling. It also allows incrementality: we could use the scenarios we developed for the challenge of MDETOOLS'18 and extend the model to handle the new system. The second extension of BP demonstrated in this paper is a set of idioms for subjecting model components to context. One of the differences between this year's challenge and the challenge we dealt with last year is that following the ball is not the only task that a player needs to handle, there is much more to care for. We demonstrate how we used the idioms for handling context to parametrize scenarios like "go to a target" in a dynamic and natural fashion such that modelers can efficiently specify reusable components similar to the way modern user manuals for advanced products are written. Lastly, in an attempt to make the instructions to the robot even more natural, we demonstrate a third extension based on deep reinforcement learning. Towards substantiating the observation that it is easier to explain things to an intelligent agent than to dumb compiler, we demonstrate how the combination of BP and deep reinforcement learning (DRL) allows for giving abstract instructions to the robot and for teaching it to follow them after a short training session.
AbstractList We describe four scenario-based implementations of controllers for a player in a simplified RoboCup-type game. All four implementations are based on the behavioural programming (BP) approach. We first describe a simple controller for the player using the state-of-the-art BPjs tool and then show how it can be extended in various ways. The first extension is based on a version of BP where the Z3 SMT solver is used to provide mechanisms for richer composition of modules within the BP model. This allows for modules with higher cohesion and lower coupling. It also allows incrementality: we could use the scenarios we developed for the challenge of MDETOOLS'18 and extend the model to handle the new system. The second extension of BP demonstrated in this paper is a set of idioms for subjecting model components to context. One of the differences between this year's challenge and the challenge we dealt with last year is that following the ball is not the only task that a player needs to handle, there is much more to care for. We demonstrate how we used the idioms for handling context to parametrize scenarios like "go to a target" in a dynamic and natural fashion such that modelers can efficiently specify reusable components similar to the way modern user manuals for advanced products are written. Lastly, in an attempt to make the instructions to the robot even more natural, we demonstrate a third extension based on deep reinforcement learning. Towards substantiating the observation that it is easier to explain things to an intelligent agent than to dumb compiler, we demonstrate how the combination of BP and deep reinforcement learning (DRL) allows for giving abstract instructions to the robot and for teaching it to follow them after a short training session.
Author Elyasaf, Achiya
Sadon, Aviran
Weiss, Gera
Yaacov, Tom
Author_xml – sequence: 1
  givenname: Achiya
  surname: Elyasaf
  fullname: Elyasaf, Achiya
  organization: Ben Gurion University of The Negev
– sequence: 2
  givenname: Aviran
  surname: Sadon
  fullname: Sadon, Aviran
  organization: Ben Gurion University of The Negev
– sequence: 3
  givenname: Gera
  surname: Weiss
  fullname: Weiss, Gera
  organization: Ben Gurion University of The Negev
– sequence: 4
  givenname: Tom
  surname: Yaacov
  fullname: Yaacov, Tom
  organization: Ben Gurion University of The Negev
BookMark eNotjF1PwjAYhWuiiYr8Am76Axj27T5oL3UgmsxA-Lgm3fYWmmzt0g10N_52IXp1Tp485zySW-ssEjICNgFg8vlzOZtnmyCdcAZywhgL5Q0ZyqmAKRcQA4_ZPRm2rclZLKIk5DE8kJ9da-yBvuJRnY07eVXRlXcHr-r6yr9Md6QbV53Rj2nqbIff3ZgqW9IZYkPXaKx2vsAabUczVN5eVxdEV5Xqr13RjambymiDJV273KWnJtj2DdKFqvGJ3GlVtTj8zwHZvc236XuQLRcf6UsWKC7CLghBykJIwRAQQPI816yMI15I0CAZTlGLi5GUpRaRzGNdlEXOyyTSiicx6HBARn-_BhH3jTe18v1eSBaJhIe_1kphgA
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/MODELS-C.2019.00039
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Xplore
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore Digital Library
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9781728151250
1728151252
EndPage 251
ExternalDocumentID 8904862
Genre orig-research
GroupedDBID 6IE
6IL
ACM
ALMA_UNASSIGNED_HOLDINGS
APO
CBEJK
LHSKQ
RIE
RIL
ID FETCH-LOGICAL-a283t-3199c8980e1e1192bbf0d542c91f190e7ef899c6ddf849b5fcdcb2d64fa2651f3
IEDL.DBID RIE
ISICitedReferencesCount 10
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000521634200029&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Sep 03 07:09:58 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a283t-3199c8980e1e1192bbf0d542c91f190e7ef899c6ddf849b5fcdcb2d64fa2651f3
PageCount 9
ParticipantIDs ieee_primary_8904862
PublicationCentury 2000
PublicationDate 2019-Sep
PublicationDateYYYYMMDD 2019-09-01
PublicationDate_xml – month: 09
  year: 2019
  text: 2019-Sep
PublicationDecade 2010
PublicationTitle 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)
PublicationTitleAbbrev MODELS-C
PublicationYear 2019
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssib058463251
ssib045274562
Score 1.849624
Snippet We describe four scenario-based implementations of controllers for a player in a simplified RoboCup-type game. All four implementations are based on the...
SourceID ieee
SourceType Publisher
StartPage 243
SubjectTerms Behavioral Programming
BPjs
Context modeling
Context Oriented Modelling
Couplings
Deep reinforcement learning
DRL
Games
Intelligent agents
Manuals
Model driven engineering
Programming
Robots
Training
Title Using Behavioural Programming with Solver, Context, and Deep Reinforcement Learning for Playing a Simplified RoboCup-Type Game
URI https://ieeexplore.ieee.org/document/8904862
WOSCitedRecordID wos000521634200029&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELVKxcAEqEV8ywNjQ-PWSey5HzBAqVqQulW2c0aVaBKVFDHx2_G5oWVgYYtOGaK7yO_u_O4dITcsTgx0mA7AJDzgKDakrLGBcuDr0gMOUaz9solkNBKzmRzXSGs7CwMAnnwGt_jo7_LT3KyxVdYWEgXi3IG7lyTxZlbr59_hkSuvop30HeJq12F3JTTEQtl-fOoPHqZBD_lcKFIZ4oLwXytVPKIMD__3LUekuRvNo-Mt6ByTGmQN8uUv_mmldYhCGvgO8q6WaMdeK53myIFuUS9H9Vm2qMpS2gco6AS8eqrxjUJaCa6-Umei4zeFY1BU0ekCqefWJax0kuu8ty4CLGHpnVpCk7wMB8-9-6BarOBCILqlO3elNEKKEBgwl-JpbcM04h0jmXUJAiRgXRlm4jS1gksdWZMa3Uljbl0II2a7J6Se5RmcEio0Y34TnnIe55GSoKUNrTBScWu4OCMN9N282GhnzCu3nf9tviAHGJwNh-uS1MvVGq7IvvkoF--rax_wb5turQ8
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTwIxEG0ImuhJDRi_7cEjK1vosu2ZDzECEsCEG2m7U0MiLEEwnvztdsoKHrx420z2sJnZ9M1M37wh5I7VYgMVpgMwMQ84ig0pa2ygHPi69IBDVNN-2UTc64nxWPZzpLSdhQEATz6De3z0d_lJatbYKisLiQJx7sDdizivhJtprZ-_h0euwIp24neIrFWH3pnUEAtlufvcaHaGQR0ZXShTGeKK8F9LVTymtI7-9zXHpLgbzqP9LeyckBzMC-TLX_3TTO0QpTTwHWRezdCO3VY6TJEFXaJekOpzVaJqntAGwIIOwOunGt8qpJnk6it1Jtp_UzgIRRUdTpF8bl3KSgepTuvrRYBFLH1QMyiSl1ZzVG8H2WoFFwRRXbmTV0ojpAiBAXNJntY2TCJeMZJZlyJADNYVYqaWJFZwqSNrEqMrSY1bF8SI2eopyc_TOZwRKjRjfheech7nkZKgpQ2tMFJxa7g4JwX03WSxUc-YZG67-Nt8Sw7ao25n0nnsPV2SQwzUhtF1RfKr5Rquyb75WE3flzc--N_HHLBW
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2019+ACM%2FIEEE+22nd+International+Conference+on+Model+Driven+Engineering+Languages+and+Systems+Companion+%28MODELS-C%29&rft.atitle=Using+Behavioural+Programming+with+Solver%2C+Context%2C+and+Deep+Reinforcement+Learning+for+Playing+a+Simplified+RoboCup-Type+Game&rft.au=Elyasaf%2C+Achiya&rft.au=Sadon%2C+Aviran&rft.au=Weiss%2C+Gera&rft.au=Yaacov%2C+Tom&rft.date=2019-09-01&rft.pub=IEEE&rft.spage=243&rft.epage=251&rft_id=info:doi/10.1109%2FMODELS-C.2019.00039&rft.externalDocID=8904862