Task Programming: Learning Data Efficient Behavior Representations

Specialized domain knowledge is often necessary to accurately annotate training sets for in-depth analysis, but can be burdensome and time-consuming to acquire from domain experts. This issue arises prominently in automated behavior analysis, in which agent movements or actions of interest are detec...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) Ročník 2021; s. 2875 - 2884
Hlavní autoři: Sun, Jennifer J., Kennedy, Ann, Zhan, Eric, Anderson, David J., Yue, Yisong, Perona, Pietro
Médium: Konferenční příspěvek Journal Article
Jazyk:angličtina
Vydáno: United States IEEE 01.06.2021
Témata:
Annotations
Mice
Programming
Training
Trajectory
Video tracking
ISSN:1063-6919, 1063-6919
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract Specialized domain knowledge is often necessary to accurately annotate training sets for in-depth analysis, but can be burdensome and time-consuming to acquire from domain experts. This issue arises prominently in automated behavior analysis, in which agent movements or actions of interest are detected from video tracking data. To reduce annotation effort, we present TREBA: a method to learn annotation-sample efficient trajectory embedding for behavior analysis, based on multi-task self-supervised learning. The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts. Total domain expert effort can be reduced by exchanging data annotation time for the construction of a small number of programmed tasks. We evaluate this trade-off using data from behavioral neuroscience, in which specialized domain knowledge is used to identify behaviors. We present experimental results in three datasets across two domains: mice and fruit flies. Using embeddings from TREBA, we reduce annotation burden by up to a factor of 10 without compromising accuracy compared to state-of-the-art features. Our results thus suggest that task programming and self-supervision can be an effective way to reduce annotation effort for domain experts.
AbstractList Specialized domain knowledge is often necessary to accurately annotate training sets for in-depth analysis, but can be burdensome and time-consuming to acquire from domain experts. This issue arises prominently in automated behavior analysis, in which agent movements or actions of interest are detected from video tracking data. To reduce annotation effort, we present TREBA: a method to learn annotation-sample efficient trajectory embedding for behavior analysis, based on multi-task self-supervised learning. The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts. Total domain expert effort can be reduced by exchanging data annotation time for the construction of a small number of programmed tasks. We evaluate this trade-off using data from behavioral neuroscience, in which specialized domain knowledge is used to identify behaviors. We present experimental results in three datasets across two domains: mice and fruit flies. Using embeddings from TREBA, we reduce annotation burden by up to a factor of 10 without compromising accuracy compared to state-of-the-art features. Our results thus suggest that task programming and self-supervision can be an effective way to reduce annotation effort for domain experts.Specialized domain knowledge is often necessary to accurately annotate training sets for in-depth analysis, but can be burdensome and time-consuming to acquire from domain experts. This issue arises prominently in automated behavior analysis, in which agent movements or actions of interest are detected from video tracking data. To reduce annotation effort, we present TREBA: a method to learn annotation-sample efficient trajectory embedding for behavior analysis, based on multi-task self-supervised learning. The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts. Total domain expert effort can be reduced by exchanging data annotation time for the construction of a small number of programmed tasks. We evaluate this trade-off using data from behavioral neuroscience, in which specialized domain knowledge is used to identify behaviors. We present experimental results in three datasets across two domains: mice and fruit flies. Using embeddings from TREBA, we reduce annotation burden by up to a factor of 10 without compromising accuracy compared to state-of-the-art features. Our results thus suggest that task programming and self-supervision can be an effective way to reduce annotation effort for domain experts.
Specialized domain knowledge is often necessary to accurately annotate training sets for in-depth analysis, but can be burdensome and time-consuming to acquire from domain experts. This issue arises prominently in automated behavior analysis, in which agent movements or actions of interest are detected from video tracking data. To reduce annotation effort, we present TREBA: a method to learn annotation-sample efficient trajectory embedding for behavior analysis, based on multi-task self-supervised learning. The tasks in our method can be efficiently engineered by domain experts through a process we call “task programming”, which uses programs to explicitly encode structured knowledge from domain experts. Total domain expert effort can be reduced by exchanging data annotation time for the construction of a small number of programmed tasks. We evaluate this trade-off using data from behavioral neuroscience, in which specialized domain knowledge is used to identify behaviors. We present experimental results in three datasets across two domains: mice and fruit flies. Using embeddings from TREBA, we reduce annotation burden by up to a factor of 10 without compromising accuracy compared to state-of-the-art features. Our results thus suggest that task programming and self-supervision can be an effective way to reduce annotation effort for domain experts.
Author Sun, Jennifer J.
Yue, Yisong
Zhan, Eric
Anderson, David J.
Kennedy, Ann
Perona, Pietro
AuthorAffiliation 2 Northwestern University
1 Caltech
AuthorAffiliation_xml – name: 1 Caltech
– name: 2 Northwestern University
Author_xml – sequence: 1
  givenname: Jennifer J.
  surname: Sun
  fullname: Sun, Jennifer J.
  email: jjsun@caltech.edu
  organization: Caltech
– sequence: 2
  givenname: Ann
  surname: Kennedy
  fullname: Kennedy, Ann
  organization: Northwestern University
– sequence: 3
  givenname: Eric
  surname: Zhan
  fullname: Zhan, Eric
  organization: Caltech
– sequence: 4
  givenname: David J.
  surname: Anderson
  fullname: Anderson, David J.
  organization: Caltech
– sequence: 5
  givenname: Yisong
  surname: Yue
  fullname: Yue, Yisong
  organization: Caltech
– sequence: 6
  givenname: Pietro
  surname: Perona
  fullname: Perona, Pietro
  organization: Caltech
BackLink https://www.ncbi.nlm.nih.gov/pubmed/36544482$$D View this record in MEDLINE/PubMed
BookMark eNpVUF1Lw0AQPKVirfYXKJJHX1Jv7ys5HwStn1BQSvU1XHKbetok9S4V_PdGqkWfdtiZnRl2QHp1UyMhR0BHAFSfjp8fp0IJnowYZTCilGm6RQaglBRCUs22yR5QxWOlQff-4D4ZhvBKKeUMQOl0l_T5941I2R65nJnwFj36Zu5NVbl6fhZN0Pi6Q9GVaU10XZaucFi30SW-mA_X-GiKS4-hW5nWNXU4IDulWQQc_sx98nRzPRvfxZOH2_vxxSR2HEQbC5MypAUtmAWUpQTDLGM8BQ1porhGLTXPbW4FJNoyKa0qOWecK2vyXBV8n5yvfZervEJbdAW8WWRL7yrjP7PGuOw_U7uXbN58ZDpRigrVGZz8GPjmfYWhzSoXClwsTI3NKmQskQkooGnaSY__Zm1Cfh_XCQ7XAoeIG1rLJNGg-BdNv36s
CODEN IEEPAD
ContentType Conference Proceeding
Journal Article
DBID 6IE
6IH
CBEJK
RIE
RIO
NPM
7X8
5PM
DOI 10.1109/CVPR46437.2021.00290
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
PubMed
MEDLINE - Academic
PubMed Central (Full Participant titles)
DatabaseTitle PubMed
MEDLINE - Academic
DatabaseTitleList MEDLINE - Academic

PubMed
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
– sequence: 3
  dbid: 7X8
  name: MEDLINE - Academic
  url: https://search.proquest.com/medline
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
Computer Science
EISBN 1665445092
9781665445092
EISSN 1063-6919
EndPage 2884
ExternalDocumentID PMC9766046
36544482
9577916
Genre orig-research
Journal Article
GrantInformation_xml – fundername: NIMH NIH HHS
  grantid: R00 MH117264
– fundername: NIMH NIH HHS
  grantid: K99 MH117264
GroupedDBID 6IE
6IH
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IJVOP
OCL
RIE
RIL
RIO
23M
29F
29O
6IK
ABDPE
ACGFS
IPLJI
M43
NPM
RIG
RNS
7X8
5PM
ID FETCH-LOGICAL-i314t-4a82e0c0c2d1e5f51a2d223819187639e9593bdbd4179d255d6f332336dabb6c3
IEDL.DBID RIE
ISICitedReferencesCount 27
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000739917303008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1063-6919
IngestDate Thu Aug 21 18:39:04 EDT 2025
Thu Oct 02 10:41:04 EDT 2025
Wed Feb 19 02:09:01 EST 2025
Wed Aug 27 02:24:15 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i314t-4a82e0c0c2d1e5f51a2d223819187639e9593bdbd4179d255d6f332336dabb6c3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
PMID 36544482
PQID 2757161088
PQPubID 23479
PageCount 10
ParticipantIDs proquest_miscellaneous_2757161088
pubmedcentral_primary_oai_pubmedcentral_nih_gov_9766046
ieee_primary_9577916
pubmed_primary_36544482
PublicationCentury 2000
PublicationDate 20210601
PublicationDateYYYYMMDD 2021-06-01
PublicationDate_xml – month: 6
  year: 2021
  text: 20210601
  day: 1
PublicationDecade 2020
PublicationPlace United States
PublicationPlace_xml – name: United States
PublicationTitle Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online)
PublicationTitleAbbrev CVPR
PublicationTitleAlternate Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit
PublicationYear 2021
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003211698
ssj0023720
Score 2.5533202
Snippet Specialized domain knowledge is often necessary to accurately annotate training sets for in-depth analysis, but can be burdensome and time-consuming to acquire...
SourceID pubmedcentral
proquest
pubmed
ieee
SourceType Open Access Repository
Aggregation Database
Index Database
Publisher
StartPage 2875
SubjectTerms Annotations
Mice
Programming
Training
Trajectory
Video tracking
Title Task Programming: Learning Data Efficient Behavior Representations
URI https://ieeexplore.ieee.org/document/9577916
https://www.ncbi.nlm.nih.gov/pubmed/36544482
https://www.proquest.com/docview/2757161088
https://pubmed.ncbi.nlm.nih.gov/PMC9766046
Volume 2021
WOSCitedRecordID wos000739917303008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT4NAEJ60jQdPVVsVHw0mHsXuA1jWo7WNB9M0TW16Iwu7aGOkplB_v7uAaE0v3kgYYDM77My3O_MNwHVECaKJmQFMPcc1RVyBQMjRzgolPJGMFk375k9sPA4WCz5pwE1dC6OUKpLP1K25LM7y5SremK2yPvcY0-FME5qMsbJWq95PoRrJ-DyoquMw4v3BfDJ1zbmURoEEm52TYuE1r90VTv7NivzlZkbt_w3wALo_9Xr2pPZEh9BQ6RG0qwDTrn7frAP3M5G9GUmTkvWuRe_sil_1xX4QubCHBaGE_oZd8Sau7WmRKltVKKVZF55Hw9ng0amaKDhLit3ccUVAFIpRTCRWXuJhQSQpcZoho-PKMBNHMpKmFZnUAEP6CaWEUl-KKPJjegytdJWqU7AV8hUlMfVpxLVPk5FCbpSwQLgx5onyLOgYlYQfJU9GWGnDgqtvZYfads2BhEjVapOFhHkarmG90FlwUiq_fpj6nquhI7GAbU1LLWB4sbfvpMvXgh9bR1i-hv1nu4dzDvvGHsp0rwto5euNuoS9-DNfZuueNq1F0CtM6wsSRM6Q
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dT8JADG8QTfQJFdT5ORMfndzXvnwUIRiREIKEt-W2uykxDsPAv9-7bU4xvPi2ZN12abtre21_BbgKKUE01hLA1LaYbuLyOEKWMlYo9mPh0mxo37jn9vveZOIPKnBd9sJIKbPiM3mjL7NcvphFS31U1vRt11XuzAZs2owRnHdrlScqVMUyju8V_XEY-c3WeDBkOjOl4kCC9dlJtvXqF69zKP_WRf4yNJ3a_5a4C42fjj1zUNqiPajIZB9qhYtpFj9wWoe7EU_fNKUuynpXpLdmgbD6Yt7zBTfbGaSE-oZZICfOzWFWLFv0KCVpA5477VGraxVjFKwpxWxhMe4RiSIUEYGlHduYE0HySE3D0flSYxOHIhR6GJlQIYZwYkoJpY7gYehE9ACqySyRR2BK5EhKIurQ0FdWTYQSsTB2Pc4i7MfSNqCuWRJ85EgZQcENAy6_mR0o7dUpCZ7I2TINiGurgA2rrc6Aw5z55cPUUVJmHjHAXRFLSaCRsVfvJNPXDCFb-ViOCvyP1y_nAra7o6de0HvoP57AjtaNvPjrFKqL-VKewVb0uZim8_NMwb4AE-zQ7w
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Task+Programming%3A+Learning+Data+Efficient+Behavior+Representations&rft.au=Sun%2C+Jennifer+J.&rft.au=Kennedy%2C+Ann&rft.au=Zhan%2C+Eric&rft.au=Anderson%2C+David+J.&rft.date=2021-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=2875&rft.epage=2884&rft_id=info:doi/10.1109%2FCVPR46437.2021.00290&rft.externalDocID=9577916
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-6919&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-6919&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-6919&client=summon