VrdONE: One-stage Video Visual Relation Detection

Video Visual Relation Detection (VidVRD) focuses on understanding how entities interact over time and space in videos, a key step for gaining deeper insights into video scenes beyond basic visual tasks. Traditional methods for VidVRD, challenged by its complexity, typically split the task into two p...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:arXiv.org
Hauptverfasser: Jiang, Xinjie, Zheng, Chenxi, Xu, Xuemiao, Liu, Bangzhen, Zheng, Weiying, Zhang, Huaidong, He, Shengfeng
Format: Paper
Sprache:Englisch
Veröffentlicht: Ithaca Cornell University Library, arXiv.org 18.08.2024
Schlagworte:
Instance segmentation
Task complexity
Visual tasks
ISSN:2331-8422
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Video Visual Relation Detection (VidVRD) focuses on understanding how entities interact over time and space in videos, a key step for gaining deeper insights into video scenes beyond basic visual tasks. Traditional methods for VidVRD, challenged by its complexity, typically split the task into two parts: one for identifying what relation categories are present and another for determining their temporal boundaries. This split overlooks the inherent connection between these elements. Addressing the need to recognize entity pairs' spatiotemporal interactions across a range of durations, we propose VrdONE, a streamlined yet efficacious one-stage model. VrdONE combines the features of subjects and objects, turning predicate detection into 1D instance segmentation on their combined representations. This setup allows for both relation category identification and binary mask generation in one go, eliminating the need for extra steps like proposal generation or post-processing. VrdONE facilitates the interaction of features across various frames, adeptly capturing both short-lived and enduring relations. Additionally, we introduce the Subject-Object Synergy (SOS) module, enhancing how subjects and objects perceive each other before combining. VrdONE achieves state-of-the-art performances on the VidOR benchmark and ImageNet-VidVRD, showcasing its superior capability in discerning relations across different temporal scales. The code is available at \textcolor[RGB]{228,58,136}{\href{https://github.com/lucaspk512/vrdone}{https://github.com/lucaspk512/vrdone}}.
AbstractList Video Visual Relation Detection (VidVRD) focuses on understanding how entities interact over time and space in videos, a key step for gaining deeper insights into video scenes beyond basic visual tasks. Traditional methods for VidVRD, challenged by its complexity, typically split the task into two parts: one for identifying what relation categories are present and another for determining their temporal boundaries. This split overlooks the inherent connection between these elements. Addressing the need to recognize entity pairs' spatiotemporal interactions across a range of durations, we propose VrdONE, a streamlined yet efficacious one-stage model. VrdONE combines the features of subjects and objects, turning predicate detection into 1D instance segmentation on their combined representations. This setup allows for both relation category identification and binary mask generation in one go, eliminating the need for extra steps like proposal generation or post-processing. VrdONE facilitates the interaction of features across various frames, adeptly capturing both short-lived and enduring relations. Additionally, we introduce the Subject-Object Synergy (SOS) module, enhancing how subjects and objects perceive each other before combining. VrdONE achieves state-of-the-art performances on the VidOR benchmark and ImageNet-VidVRD, showcasing its superior capability in discerning relations across different temporal scales. The code is available at \textcolor[RGB]{228,58,136}{\href{https://github.com/lucaspk512/vrdone}{https://github.com/lucaspk512/vrdone}}.
Author Liu, Bangzhen
Jiang, Xinjie
Zheng, Chenxi
Xu, Xuemiao
Zhang, Huaidong
Zheng, Weiying
He, Shengfeng
Author_xml – sequence: 1
  givenname: Xinjie
  surname: Jiang
  fullname: Jiang, Xinjie
– sequence: 2
  givenname: Chenxi
  surname: Zheng
  fullname: Zheng, Chenxi
– sequence: 3
  givenname: Xuemiao
  surname: Xu
  fullname: Xu, Xuemiao
– sequence: 4
  givenname: Bangzhen
  surname: Liu
  fullname: Liu, Bangzhen
– sequence: 5
  givenname: Weiying
  surname: Zheng
  fullname: Zheng, Weiying
– sequence: 6
  givenname: Huaidong
  surname: Zhang
  fullname: Zhang, Huaidong
– sequence: 7
  givenname: Shengfeng
  surname: He
  fullname: He, Shengfeng
BookMark eNotj81Kw0AUhQdRsNY-gLuA68Sbe2c6E3dSWxVKA6V0WybJHUkJE80k4uM7opvvnNX5uRGXvvcsxF0OmTRKwYMdvtuvDCWYDIrICzFDojw1EvFaLEI4AwAuNSpFM5Efh6bcrR-T0nMaRvvOybFtuI8Mk-2SPXd2bHufPPPI9a-7FVfOdoEX_zoXh836sHpNt-XL2-ppm1qFmCoy0hHoSktGZw3IAli7RgIb69gwEynN3LCTpnDW4RJqkiR1VdnaNDQX93-xH0P_OXEYT-d-GnxsPFH8VVDcj_QDmUNGsg
ContentType Paper
Copyright 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.2408.09408
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials
ProQuest Central
Technology Collection
ProQuest One Community College
ProQuest Central
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
ProQuest Central Premium
ProQuest One Academic
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a522-5384f307b74e2fa80490e7fd40e8afe8ee3357eedef489faf260c34347bbac8d3
IEDL.DBID M7S
IngestDate Mon Jun 30 09:19:57 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a522-5384f307b74e2fa80490e7fd40e8afe8ee3357eedef489faf260c34347bbac8d3
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/3094932552?pq-origsite=%requestingapplication%
PQID 3094932552
PQPubID 2050157
ParticipantIDs proquest_journals_3094932552
PublicationCentury 2000
PublicationDate 20240818
PublicationDateYYYYMMDD 2024-08-18
PublicationDate_xml – month: 08
  year: 2024
  text: 20240818
  day: 18
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2024
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.8804979
SecondaryResourceType preprint
Snippet Video Visual Relation Detection (VidVRD) focuses on understanding how entities interact over time and space in videos, a key step for gaining deeper insights...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Instance segmentation
Task complexity
Visual tasks
Title VrdONE: One-stage Video Visual Relation Detection
URI https://www.proquest.com/docview/3094932552
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEA7aKnjyjY9a9uB1-9hkN1kvgtqiB7eLllJPJdlMpCDbutsWf76TuNWD4MVLIIRAkpnM82OGkEvDM9SqXPkMInRQqJG-7CIvZ6hsmNHcgDau2QRPEjEex2kVcCsrWOVaJjpBrWeZjZG3KfohaGuEYXA9f_dt1yibXa1aaGySuq2SEDjo3vN3jCWIOG6gX8lMV7qrLYuP6apl63q1bOU48UsEO73S3_3vifZIPZVzKPbJBuQHZNvhObPykHRHhR4kvStvkIOPBuAreKOphhmO5VK-eWsMnHcHCwfGyo_IsN8b3t77VXcEX4bWgaSCGfygijMIjBQ2gwfcaNYBIQ0IAEpDjhoQDBOxkQYdl4wyyrhSMhOaHpNaPsvhhHgdE4FRMRoy0iDJQgU604pFVKu4y0J2ShrrB5hUHF5Ofm5_9vfyOdkJ0BCwcdiuaJDaoljCBdnKVotpWTRJ_aaXpE9NRzicpQ-P6csnwO2jpw
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LS8QwEB58oiff-Fi1Bz121zZpkwriwQfKrlVwWfZW0mYiC9LVdn39KP-jk7rVg-DNg5dcCiFMvs7zywzAnhEZWVWRuhxDClCYUa7yCMsZGRtutDCoTTVsQsSx7Pejmwl4r9_CWFplrRMrRa2Hmc2RtxjFIeRrBIF__PDo2qlRtrpaj9D4hEUb314oZCuPLk_pfvd9__yse3LhjqcKuCqwgReT3BCwU8HRN0rayhcKo_kBSmVQIjIWCLIcaLiMjDLk8GeMMy7SVGVSM9p2Eqa5Vf4VU_D2K6Xjh4LOxz5rp1WnsJYqXgfPTdtGrGkb1ckfGr8yY-cL_0wAizB9ox6wWIIJzJdhtmKrZuUKeL1CX8dnh851ji65t3fo9AYah7SWT-reqRl-zimOKqpZvgrdvzjkGkzlwxzXwTkwIZo0IjdNGQJkkKLOdMpDptPI4wHfgEYt72T8_5bJt7A3f_-8C3MX3atO0rmM21sw75PLYzPOnmzA1Kh4wm2YyZ5Hg7LYqbDiQPLHV_MBBOr-Bg
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=VrdONE%3A+One-stage+Video+Visual+Relation+Detection&rft.jtitle=arXiv.org&rft.au=Jiang%2C+Xinjie&rft.au=Zheng%2C+Chenxi&rft.au=Xu%2C+Xuemiao&rft.au=Liu%2C+Bangzhen&rft.date=2024-08-18&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2408.09408