Spatio-Temporal Consistent Semantic Mapping for Robotics Fruit Growth Monitoring

Automatic fruit growth monitoring plays a vital role in advancing precision agriculture. Tracking the evolution of fruits over time is essential to monitor their development and optimize production. The ability to recognize fruits over periods of time, even with drastic scene changes, is a required...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE robotics and automation letters Ročník 10; číslo 9; s. 9470 - 9477
Hlavní autori:	Lobefaro, Luca, Sodano, Matteo, Fusaro, Daniel, Magistri, Federico, Malladi, Meher V. R., Guadagnino, Tiziano, Pretto, Alberto, Stachniss, Cyrill
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Piscataway IEEE 01.09.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Annotations Cameras Computational modeling Fruits Image segmentation Instance segmentation Mapping Monitoring Point cloud compression Robotics robotics and automation in agriculture and forestry Robots Semantics Solid modeling Three dimensional models Three-dimensional displays Tracking
ISSN:	2377-3766, 2377-3766
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	Automatic fruit growth monitoring plays a vital role in advancing precision agriculture. Tracking the evolution of fruits over time is essential to monitor their development and optimize production. The ability to recognize fruits over periods of time, even with drastic scene changes, is a required capability of agricultural robots. This letter presents a system that allows long-term fruit tracking in 3D data. It generates instance-segmented 3D representations of plants at various growth stages over time, utilizing only consumer-grade RGB-D cameras installed on a mobile robot. Our approach first performs instance segmentation on each image in a sequence. Then, by exploiting geometric information and depth maps, we track the same instances throughout the sequence. We produce a 3D point cloud containing instances, exploiting odometry information and 3D semantic mapping. Once our robot performs a new recording at a different plant growth stage, it associates each fruit with the previously built 3D cloud and update the model. We validate the system in a real-world glasshouse environment in Bonn, Germany. Experimental results demonstrate that our system outperforms existing baselines even though it relies only on annotated images and operates at frame-rate, allowing the deployment on a real robot.
AbstractList	Automatic fruit growth monitoring plays a vital role in advancing precision agriculture. Tracking the evolution of fruits over time is essential to monitor their development and optimize production. The ability to recognize fruits over periods of time, even with drastic scene changes, is a required capability of agricultural robots. This letter presents a system that allows long-term fruit tracking in 3D data. It generates instance-segmented 3D representations of plants at various growth stages over time, utilizing only consumer-grade RGB-D cameras installed on a mobile robot. Our approach first performs instance segmentation on each image in a sequence. Then, by exploiting geometric information and depth maps, we track the same instances throughout the sequence. We produce a 3D point cloud containing instances, exploiting odometry information and 3D semantic mapping. Once our robot performs a new recording at a different plant growth stage, it associates each fruit with the previously built 3D cloud and update the model. We validate the system in a real-world glasshouse environment in Bonn, Germany. Experimental results demonstrate that our system outperforms existing baselines even though it relies only on annotated images and operates at frame-rate, allowing the deployment on a real robot.
Author	Sodano, Matteo Magistri, Federico Lobefaro, Luca Guadagnino, Tiziano Pretto, Alberto Fusaro, Daniel Stachniss, Cyrill Malladi, Meher V. R.
Author_xml	– sequence: 1 givenname: Luca orcidid: 0009-0009-8391-8776 surname: Lobefaro fullname: Lobefaro, Luca email: llobefar@uni-bonn.de organization: Center for Robotics, University of Bonn, Bonn, Germany – sequence: 2 givenname: Matteo orcidid: 0000-0003-3358-6979 surname: Sodano fullname: Sodano, Matteo organization: Center for Robotics, University of Bonn, Bonn, Germany – sequence: 3 givenname: Daniel orcidid: 0009-0000-4053-5577 surname: Fusaro fullname: Fusaro, Daniel organization: University of Padua, Padua, Italy – sequence: 4 givenname: Federico orcidid: 0000-0003-2815-5760 surname: Magistri fullname: Magistri, Federico organization: Center for Robotics, University of Bonn, Bonn, Germany – sequence: 5 givenname: Meher V. R. orcidid: 0000-0002-2929-4933 surname: Malladi fullname: Malladi, Meher V. R. organization: Center for Robotics, University of Bonn, Bonn, Germany – sequence: 6 givenname: Tiziano orcidid: 0000-0002-7853-5510 surname: Guadagnino fullname: Guadagnino, Tiziano organization: Center for Robotics, University of Bonn, Bonn, Germany – sequence: 7 givenname: Alberto orcidid: 0000-0003-1920-2887 surname: Pretto fullname: Pretto, Alberto organization: University of Padua, Padua, Italy – sequence: 8 givenname: Cyrill orcidid: 0000-0003-1173-6972 surname: Stachniss fullname: Stachniss, Cyrill organization: Center for Robotics, University of Bonn, Bonn, Germany
BookMark	eNpNkE1LAzEQhoMoWGvvHjwEPG-dfG3MsRStQovS1vOS3WQ1pU3WJEX8925pQecyw_DMvPBcoXMfvEXohsCYEFD38-VkTIGKMROKqwdxhgaUSVkwWZbn_-ZLNEppAwBEUMmUGKC3VaezC8Xa7roQ9RZPg08uZeszXtmd9tk1eKG7zvkP3IaIl6EO_S7hp7h3Gc9i-M6feBG8yyH20DW6aPU22dGpD9H70-N6-lzMX2cv08m8aCiXuShrwVtpjKrbpqR1Q5SRoAVYIDWrdWMMN6XhnKmWgDWNAiIF1NxQrgCUZEN0d_zbxfC1tylXm7CPvo-sWJ9ABJdAewqOVBNDStG2VRfdTsefikB1UFf16qqDuuqkrj-5PZ44a-0f3hcVpWK_91BsKg
CODEN	IRALC6
Cites_doi	10.1109/IROS58592.2024.10802321 10.1109/LRA.2022.3193239 10.3390/robotics10020052 10.1109/ICRA48891.2023.10160709 10.1002/nav.3800020109 10.1109/LRA.2023.3236571 10.1109/ICRA48506.2021.9562047 10.1109/34.121791 10.1109/TCYB.2013.2265378 10.1109/IROS45743.2020.9340918 10.1109/IROS58592.2024.10802065 10.1109/LRA.2019.2901987 10.1109/ICRA48891.2023.10160315 10.1109/IROS55552.2023.10342449 10.1109/ICCV51070.2023.00371 10.1109/ICICML57342.2022.10009765 10.1109/CVPR.2016.91 10.1109/ICRA48891.2023.10161350 10.1109/TPAMI.2024.3419548 10.1109/ICRA.2017.7989447 10.1109/ICRA48891.2023.10160918 10.1016/S0065-2113(08)60513-1 10.1109/ACCESS.2019.2942144 10.1109/CVPRW.2018.00060 10.1109/DICTA51227.2020.9363407 10.1016/j.atech.2024.100449 10.1109/LRA.2023.3338515 10.1017/CBO9780511811685
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/LRA.2025.3594985
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2377-3766
EndPage	9477
ExternalDocumentID	10_1109_LRA_2025_3594985 11112569
Genre	orig-research
GrantInformation_xml	– fundername: European Union's Horizon Europe Research and Innovation Programme grantid: 101070405 – fundername: Germany's Excellence Strategy – fundername: DFG, German Research Foundation
GroupedDBID	0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c247t-6b54f7dd9bfc62bc19d70a50e01b3bacdd4d6d4439f10edc901750b4d24900973
IEDL.DBID	RIE
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001550841500007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2377-3766
IngestDate	Sat Nov 22 13:40:59 EST 2025 Sat Nov 29 07:37:17 EST 2025 Wed Aug 20 06:21:05 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	9
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c247t-6b54f7dd9bfc62bc19d70a50e01b3bacdd4d6d4439f10edc901750b4d24900973
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0009-0009-8391-8776 0000-0002-2929-4933 0009-0000-4053-5577 0000-0003-2815-5760 0000-0002-7853-5510 0000-0003-3358-6979 0000-0003-1173-6972 0000-0003-1920-2887
PQID	3247154702
PQPubID	4437225
PageCount	8
ParticipantIDs	ieee_primary_11112569 crossref_primary_10_1109_LRA_2025_3594985 proquest_journals_3247154702
PublicationCentury	2000
PublicationDate	2025-09-01
PublicationDateYYYYMMDD	2025-09-01
PublicationDate_xml	– month: 09 year: 2025 text: 2025-09-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE robotics and automation letters
PublicationTitleAbbrev	LRA
PublicationYear	2025
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref30 ref11 ref10 ref1 ref17 ref16 ref19 ref18 Fusaro (ref6) 2024 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 ref5 Carlone (ref2) 2015
References_xml	– ident: ref15 doi: 10.1109/IROS58592.2024.10802321 – ident: ref18 doi: 10.1109/LRA.2022.3193239 – ident: ref20 doi: 10.3390/robotics10020052 – ident: ref10 doi: 10.1109/ICRA48891.2023.10160709 – ident: ref12 doi: 10.1002/nav.3800020109 – ident: ref28 doi: 10.1109/LRA.2023.3236571 – ident: ref25 doi: 10.1109/ICRA48506.2021.9562047 – ident: ref1 doi: 10.1109/34.121791 – ident: ref8 doi: 10.1109/TCYB.2013.2265378 – ident: ref17 doi: 10.1109/IROS45743.2020.9340918 – ident: ref19 doi: 10.1109/IROS58592.2024.10802065 – ident: ref14 doi: 10.1109/LRA.2019.2901987 – ident: ref27 doi: 10.1109/ICRA48891.2023.10160315 – ident: ref16 doi: 10.1109/IROS55552.2023.10342449 – ident: ref11 doi: 10.1109/ICCV51070.2023.00371 – ident: ref30 doi: 10.1109/ICICML57342.2022.10009765 – ident: ref22 doi: 10.1109/CVPR.2016.91 – ident: ref23 doi: 10.1109/ICRA48891.2023.10161350 – ident: ref29 doi: 10.1109/TPAMI.2024.3419548 – ident: ref5 doi: 10.1109/ICRA.2017.7989447 – ident: ref24 doi: 10.1109/ICRA48891.2023.10160918 – ident: ref21 doi: 10.1016/S0065-2113(08)60513-1 – start-page: 1 volume-title: Proc. ICRA Workshop Robot. Agriculture year: 2015 ident: ref2 article-title: Towards 4D crop analysis in precision agriculture: Estimating plant height and crown radius over time via expectation-maximization – ident: ref13 doi: 10.1109/ACCESS.2019.2942144 – ident: ref4 doi: 10.1109/CVPRW.2018.00060 – ident: ref7 doi: 10.1109/DICTA51227.2020.9363407 – ident: ref3 doi: 10.1016/j.atech.2024.100449 – ident: ref26 doi: 10.1109/LRA.2023.3338515 – year: 2024 ident: ref6 article-title: Horticultural temporal fruit monitoring via 3D instance segmentation and re-identification using point clouds – ident: ref9 doi: 10.1017/CBO9780511811685
SSID	ssj0001527395
Score	2.3088946
Snippet	Automatic fruit growth monitoring plays a vital role in advancing precision agriculture. Tracking the evolution of fruits over time is essential to monitor...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Index Database Publisher
StartPage	9470
SubjectTerms	Annotations Cameras Computational modeling Fruits Image segmentation Instance segmentation Mapping Monitoring Point cloud compression Robotics robotics and automation in agriculture and forestry Robots Semantics Solid modeling Three dimensional models Three-dimensional displays Tracking
Title	Spatio-Temporal Consistent Semantic Mapping for Robotics Fruit Growth Monitoring
URI	https://ieeexplore.ieee.org/document/11112569 https://www.proquest.com/docview/3247154702
Volume	10
WOSCitedRecordID	wos001550841500007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE/IET Electronic Library (IEL) customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwED3RigEGvhGFUnlgYUjrJk4cjxVqYaBV1RapW-SvQCVIUJsy8tuxnVSthBjYMiRW9OzzvbPv3gHcpYGOrc6bFyitPUI19mJOygIfJkIT5frcNZugo1E8n7NxVazuamG01i75TLfto7vLV7lc26OyjjVv46JZDWqU0rJYa3ugYqXEWLi5isSs8zzpmQDQD9tByAiz3ZJ3XI_rpfJrA3ZeZXD8z_85gaOKPqJeOd-nsKezMzjcERU8h_HUJUl7s1J06h25npxmMrMCTfWHQXIh0ZBbXYZXZCgrmuQit2LNaLBcLwr0aALz4g2Vxm6HvICXQX_28ORVfRM86RNaeJEISUqVYiKVkS9klymKeYg17opAcKkUUZEihoqkXayVNJTA8AZBlAnFnHzPJdSzPNNXgKSMYo5pakiXtFr0QsUsCkMepYIFKeMNuN9AmnyW8hiJCyswSwz8iYU_qeBvwIWFcPtehV4DmptJSCoDWiVm_VDD7ij2r__47AYO7OhlvlcT6sVyrW9hX34Vi9WyBbXhd7_lVsgPM9u6Ig
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8IwFH5RNFEP_sSIovbgxcOg29ptPRIjYgRCABNuy_pjSqLMwPDvt-1GIDEevO2wdcv3-va-1_Z9D-Au9VVkdN4cXyrlkFBhJ0pIUeDDONVZrpfYZhNhvx9NJmxQFqvbWhillD18phrm0u7ly0wszVJZ07i3DtFsG3YoIZ5blGutl1SMmBijq81IzJrdYUungB5t-JQRZvolbwQf203l1y_YxpX20T-_6BgOSwKJWoXFT2BLzU7hYENW8AwGI3tM2hkXslMfyHbl1Oac5WikPjWWU4F6iVFmeEOatKJhxjMj14za8-U0R086Nc_fUeHuZsgqvLYfxw8dp-yc4AiPhLkTcErSUErGUxF4XLhMhjihWGGX-zwRUhIZSKLJSOpiJYUmBZo5cCJ1MmYFfM6hMstm6gKQEEGU4DDVtEsYNXouIxZQmgQpZ37KkhrcryCNvwqBjNgmFpjFGv7YwB-X8NegaiBc31eiV4P6yghx6UKLWM-gUPO7EHuXfzx2C3udca8bd5_7L1ewb95UnP6qQyWfL9U17IrvfLqY39h58gOjQrw4
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Spatio-Temporal+Consistent+Semantic+Mapping+for+Robotics+Fruit+Growth+Monitoring&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Lobefaro%2C+Luca&rft.au=Sodano%2C+Matteo&rft.au=Fusaro%2C+Daniel&rft.au=Magistri%2C+Federico&rft.date=2025-09-01&rft.pub=IEEE&rft.eissn=2377-3766&rft.volume=10&rft.issue=9&rft.spage=9470&rft.epage=9477&rft_id=info:doi/10.1109%2FLRA.2025.3594985&rft.externalDocID=11112569
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon