High Precision Leaf Instance Segmentation for Phenotyping in Point Clouds Obtained Under Real Field Conditions

Measuring plant traits with high throughput allows breeders to monitor and select the best cultivars for subsequent breeding generations. This can enable farmers to improve yield to produce more food, feed, and fiber. Current breeding practices involve extracting leaf parameters on a small subset of...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE robotics and automation letters Jg. 8; H. 8; S. 4791 - 4798
Hauptverfasser:	Marks, Elias, Sodano, Matteo, Magistri, Federico, Wiesmann, Louis, Desai, Dhagash, Marcuzzi, Rodrigo, Behley, Jens, Stachniss, Cyrill
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Piscataway IEEE 01.08.2023 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Agricultural automation Artificial neural networks Crops Deep learning deep learning for visual perception Encoding Image segmentation Instance segmentation Neural networks Physical work Point cloud compression robotics and automation in agriculture and forestry Task analysis
ISSN:	2377-3766, 2377-3766
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	Measuring plant traits with high throughput allows breeders to monitor and select the best cultivars for subsequent breeding generations. This can enable farmers to improve yield to produce more food, feed, and fiber. Current breeding practices involve extracting leaf parameters on a small subset of the leaves present in the breeding plots, while still requiring substantial manual labor. To automate this process, an important step is the precise distinction between separate leaves, which is the problem we address in this letter. We exploit recent advancements in 3D deep learning to build a convolutional neural network that learns to segment individual leaves. As done in current breeding practices, we select a subset of leaves to be used for phenotypic trait evaluation as this allows us to alleviate the influence of segmentation errors on the phenotypic trait estimation. To this extent we propose to use an additional neural network to predict the quality of each segmented leaf and discard inaccurate leaf instances. The experiments show that our network yields higher segmentation accuracy on sugar beet breeding plots planted under the supervision of the German Federal Office for Plant Varieties. Furthermore, we show that our neural network helps in filtering out leaves with lower segmentation accuracy.
AbstractList	Measuring plant traits with high throughput allows breeders to monitor and select the best cultivars for subsequent breeding generations. This can enable farmers to improve yield to produce more food, feed, and fiber. Current breeding practices involve extracting leaf parameters on a small subset of the leaves present in the breeding plots, while still requiring substantial manual labor. To automate this process, an important step is the precise distinction between separate leaves, which is the problem we address in this letter. We exploit recent advancements in 3D deep learning to build a convolutional neural network that learns to segment individual leaves. As done in current breeding practices, we select a subset of leaves to be used for phenotypic trait evaluation as this allows us to alleviate the influence of segmentation errors on the phenotypic trait estimation. To this extent we propose to use an additional neural network to predict the quality of each segmented leaf and discard inaccurate leaf instances. The experiments show that our network yields higher segmentation accuracy on sugar beet breeding plots planted under the supervision of the German Federal Office for Plant Varieties. Furthermore, we show that our neural network helps in filtering out leaves with lower segmentation accuracy.
Author	Sodano, Matteo Behley, Jens Magistri, Federico Wiesmann, Louis Desai, Dhagash Marks, Elias Stachniss, Cyrill Marcuzzi, Rodrigo
Author_xml	– sequence: 1 givenname: Elias orcidid: 0000-0003-1322-9317 surname: Marks fullname: Marks, Elias email: elias.marks@uni-bonn.de organization: University of Bonn, Bonn, Germany – sequence: 2 givenname: Matteo orcidid: 0000-0003-3358-6979 surname: Sodano fullname: Sodano, Matteo email: matteo.sodano7@gmail.com organization: University of Bonn, Bonn, Germany – sequence: 3 givenname: Federico orcidid: 0000-0003-2815-5760 surname: Magistri fullname: Magistri, Federico email: federico.magistri@igg.uni-bonn.de organization: University of Bonn, Bonn, Germany – sequence: 4 givenname: Louis orcidid: 0000-0003-0985-7433 surname: Wiesmann fullname: Wiesmann, Louis email: louis.wiesmann@igg.uni-bonn.de organization: University of Bonn, Bonn, Germany – sequence: 5 givenname: Dhagash orcidid: 0009-0006-5475-4934 surname: Desai fullname: Desai, Dhagash email: desaidhagash@gmail.com organization: University of Bonn, Bonn, Germany – sequence: 6 givenname: Rodrigo orcidid: 0000-0001-8076-0293 surname: Marcuzzi fullname: Marcuzzi, Rodrigo email: rodrigomarcuzzi@gmail.com organization: University of Bonn, Bonn, Germany – sequence: 7 givenname: Jens orcidid: 0000-0001-6483-0319 surname: Behley fullname: Behley, Jens email: jens.behley@igg.uni-bonn.de organization: University of Bonn, Bonn, Germany – sequence: 8 givenname: Cyrill orcidid: 0000-0003-1173-6972 surname: Stachniss fullname: Stachniss, Cyrill email: cyrill.stachniss@igg.uni-bonn.de organization: University of Bonn, Bonn, Germany
BookMark	eNp9kL1vwjAQxa2qlUope4cOljpD_ZHE8YhQKUhIIFrmyIkvYBRsapuB_76JYEAdOt1J937v7t4TurfOAkIvlIwoJfJ9sR6PGGF8xFme85zfoR7jQgy5yLL7m_4RDULYE0JoygSXaQ_Zmdnu8MpDZYJxFi9A1XhuQ1S2AvwF2wPYqGI3qp3Hqx1YF89HY7fYWLxyxkY8adxJB7wsozIWNN5YDR6vQTV4aqDReOKsNp1HeEYPtWoCDK61jzbTj-_JbLhYfs4n48WwYpLFYQ1cViBKrZkusySlEojM29NoVjKag0gk1CXoVLJE1jTNOOOU1EmpSMqU1ryP3i6-R-9-ThBisXcnb9uVBcs5ZyIRImtV2UVVeReCh7qozOXZ6JVpCkqKLt6ijbfo4i2u8bYg-QMevTkof_4Peb0gBgBu5DTNheT8F2x-iAw
CODEN	IRALC6
CitedBy_id	crossref_primary_10_1016_j_jafr_2025_102343 crossref_primary_10_1016_j_plaphe_2025_100071 crossref_primary_10_1016_j_aei_2025_103199 crossref_primary_10_1016_j_plaphe_2025_100107 crossref_primary_10_1109_LRA_2023_3320018 crossref_primary_10_1109_JSTARS_2023_3312815 crossref_primary_10_1109_TRO_2025_3567544 crossref_primary_10_1016_j_biosystemseng_2025_01_016
Cites_doi	10.1109/CVPR42600.2020.00492 10.1109/ICCV.2019.00939 10.1007/978-3-030-89177-0_4 10.1109/CVPR52688.2022.00273 10.1109/ICRA48506.2021.9561356 10.1109/LRA.2022.3147462 10.1109/ICCV.2019.00651 10.1111/nph.15817 10.1146/annurev-arplant-050312-120137 10.1109/CVPR.2019.00963 10.1109/CVPR.2019.00319 10.1109/ICCVW54120.2021.00145 10.1109/LRA.2022.3193239 10.1016/j.biosystemseng.2019.08.014 10.1371/journal.pone.0256340 10.1109/CVPR42600.2020.01249 10.1109/ICCV.2015.114 10.1109/CVPR.2017.261 10.1109/ICCV.2017.322 10.1109/ICDMW.2017.12 10.1109/BHI.2018.8333411 10.1109/LRA.2023.3236568 10.1109/ICRA46639.2022.9811358 10.1109/WACV51458.2022.00302 10.1109/CVPR.2016.170
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/LRA.2023.3288383
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Xplore CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2377-3766
EndPage	4798
ExternalDocumentID	10_1109_LRA_2023_3288383 10158793
Genre	orig-research
GrantInformation_xml	– fundername: Bundesministerium für Ernährung und Landwirtschaft; Federal Ministry of Food and Agriculture grantid: 28DK108B20 funderid: 10.13039/501100005908 – fundername: Deutsche Forschungsgemeinschaft; German Research Foundation grantid: EXC-2070 – 390732324 funderid: 10.13039/501100001659 – fundername: Deutsche Forschungsgemeinschaft funderid: 10.13039/501100001659
GroupedDBID	0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c292t-fe39ce7bdd2db64519e098eaf16b218e749efbed59249f15632310f4ba052add3
IEDL.DBID	RIE
ISICitedReferencesCount	11
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001024186200004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2377-3766
IngestDate	Sun Nov 30 04:40:36 EST 2025 Sat Nov 29 06:03:25 EST 2025 Tue Nov 18 21:00:09 EST 2025 Wed Aug 27 02:14:22 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	8
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c292t-fe39ce7bdd2db64519e098eaf16b218e749efbed59249f15632310f4ba052add3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-1322-9317 0000-0003-3358-6979 0000-0003-1173-6972 0000-0001-6483-0319 0000-0003-2815-5760 0009-0006-5475-4934 0000-0001-8076-0293 0000-0003-0985-7433
PQID	2833274776
PQPubID	4437225
PageCount	8
ParticipantIDs	proquest_journals_2833274776 ieee_primary_10158793 crossref_primary_10_1109_LRA_2023_3288383 crossref_citationtrail_10_1109_LRA_2023_3288383
PublicationCentury	2000
PublicationDate	2023-08-01
PublicationDateYYYYMMDD	2023-08-01
PublicationDate_xml	– month: 08 year: 2023 text: 2023-08-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE robotics and automation letters
PublicationTitleAbbrev	LRA
PublicationYear	2023
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref31 ref30 ref11 ref10 ref2 ref1 ref17 ref16 roggiolani (ref23) 0 ref19 roggiolani (ref21) 0 ref18 schult (ref24) 0 ref26 ref25 roggiolani (ref22) 0 ref28 ref27 commission (ref6) 2018 ref29 ref8 ref7 ref9 ref4 ref3 ref5 qi (ref20) 0
References_xml	– ident: ref13 doi: 10.1109/CVPR42600.2020.00492 – ident: ref2 doi: 10.1109/ICCV.2019.00939 – year: 0 ident: ref23 article-title: Unsupervised pre-training for leaf instance segmentation in 3D publication-title: Under Review – ident: ref12 doi: 10.1007/978-3-030-89177-0_4 – ident: ref29 doi: 10.1109/CVPR52688.2022.00273 – ident: ref15 doi: 10.1109/ICRA48506.2021.9561356 – ident: ref31 doi: 10.1109/LRA.2022.3147462 – ident: ref28 doi: 10.1109/ICCV.2019.00651 – year: 2018 ident: ref6 article-title: Protocol for tests on distinctness, uniformity and stability publication-title: Beta Vulgaris l ssp Vulgaris var Altissima Döll – ident: ref9 doi: 10.1111/nph.15817 – ident: ref8 doi: 10.1146/annurev-arplant-050312-120137 – ident: ref14 doi: 10.1109/CVPR.2019.00963 – start-page: 652 year: 0 ident: ref20 article-title: PointNet: Deep learning on point sets for 3D classification and segmentation publication-title: Proc IEEE Conf Comput Vis Pattern Recognit – ident: ref5 doi: 10.1109/CVPR.2019.00319 – ident: ref10 doi: 10.1109/ICCVW54120.2021.00145 – year: 0 ident: ref24 article-title: Mask3D for 3D semantic instance segmentation publication-title: Proc IEEE Int Conf Robot Automat – year: 0 ident: ref21 article-title: On domain-specific pre-training for effective semantic perception in agricultural robotics publication-title: Proc IEEE Int Conf Robot Automat – ident: ref16 doi: 10.1109/LRA.2022.3193239 – ident: ref26 doi: 10.1016/j.biosystemseng.2019.08.014 – year: 0 ident: ref22 article-title: Hierarchical approach for joint semantic, plant instance, and leaf instance segmentation in the agricultural domain publication-title: Proc IEEE Int Conf Robot Automat – ident: ref25 doi: 10.1371/journal.pone.0256340 – ident: ref4 doi: 10.1109/CVPR42600.2020.01249 – ident: ref27 doi: 10.1109/ICCV.2015.114 – ident: ref7 doi: 10.1109/CVPR.2017.261 – ident: ref11 doi: 10.1109/ICCV.2017.322 – ident: ref19 doi: 10.1109/ICDMW.2017.12 – ident: ref3 doi: 10.1109/BHI.2018.8333411 – ident: ref17 doi: 10.1109/LRA.2023.3236568 – ident: ref18 doi: 10.1109/ICRA46639.2022.9811358 – ident: ref30 doi: 10.1109/WACV51458.2022.00302 – ident: ref1 doi: 10.1109/CVPR.2016.170
SSID	ssj0001527395
Score	2.3254468
Snippet	Measuring plant traits with high throughput allows breeders to monitor and select the best cultivars for subsequent breeding generations. This can enable...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	4791
SubjectTerms	Agricultural automation Artificial neural networks Crops Deep learning deep learning for visual perception Encoding Image segmentation Instance segmentation Neural networks Physical work Point cloud compression robotics and automation in agriculture and forestry Task analysis
Title	High Precision Leaf Instance Segmentation for Phenotyping in Point Clouds Obtained Under Real Field Conditions
URI	https://ieeexplore.ieee.org/document/10158793 https://www.proquest.com/docview/2833274776
Volume	8
WOSCitedRecordID	wos001024186200004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Xplore customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED5RxAADb0ShIA8sDCkhL8cjqlqBVKDiJbYosc9QqSSoTZFY-O34nJSHEEhsGezI8ne2v7Pv7gM4wEy7GGrPCXiUOkGqUydGqR0MpIyFUqm0Mp13fX5xEd_fi0GdrG5zYRDRBp9hmz7tW74q5JSuyswKPw5jY1ANaHDOq2StzwsVKiUmwtlTpCuO-lcnbVIHb_skqRv7344eq6XyYwO2p0pv5Z_jWYXlmj6ykwrvNZjDfB2WvhQV3ICcQjfYYFyL57A-ppqdWRYokV3jw1OdbpQzQ1jZ4BHzonyltCk2zNmgGOYl64yKqZqwy4wuDlAxq47ErgypZD0KeWOdgp66yWQ34bbXvemcOrWqgiM94ZWORl9I5JlSnsoiqi6DrojNUI6jzJz3yAOBOkMVkmemjXvnEwXUQZa6oWd2Q38L5vMix21goSE_qdkxA-PDBDrgWRQqM9fSeB2GWMRpE45mE57IuuQ4KV-MEut6uCIxECUEUVJD1ITDjx7PVbmNP9puEiRf2lVoNKE1AzWpF-QkMSzKJwecRzu_dNuFRfp7FdzXgvlyPMU9WJAv5XAy3ofG-Vt331rcOx1r1P8
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT-MwEB4tD4nlwLMrCiz4wIVDSpo4Dx9RRQXaUCpe4hYl9hgqQYLaFGn__XqclIcQSHvLwVYsf2P7G3tmPoADzLWLgfYcHoWZwzOdOTFK7SCXMhZKZdLKdN4m0WAQ392JYZOsbnNhENEGn2GHPu1bvirllK7KzArvBrExqDlYCDj3unW61tuVChUTE8HsMdIVR8nlcYf0wTs-ierG_ofDx6qpfNqC7bnSX_3PEa3BSkMg2XGN-Dr8wGIDlt-VFdyEgoI32HDcyOewBDPNziwPlMiu8P6pSTgqmKGsbPiARVn9pcQpNirYsBwVFes9llM1YRc5XR2gYlYfiV0aWsn6FPTGeiU9dpPRtuCmf3LdO3UaXQVHesKrHI2-kBjlSnkqD6m-DLoiNkPphrk58THiAnWOKiDfTBsHzycSqHmeuYFn9kP_F8wXZYFbwAJDfzKzZxo0Yq55lIeBMnMtjd9hqEWcteFoNuGpbIqOk_bFY2qdD1ekBqKUIEobiNpw-NrjuS648U3bFkHyrl2NRht2Z6CmzZKcpIZH-eSCR-H2F932Yen0-jxJk7PBnx34SX-qQ_12Yb4aT_E3LMqXajQZ71m7-wdO-dcV
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=High+Precision+Leaf+Instance+Segmentation+for+Phenotyping+in+Point+Clouds+Obtained+Under+Real+Field+Conditions&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Marks%2C+Elias&rft.au=Sodano%2C+Matteo&rft.au=Magistri%2C+Federico&rft.au=Wiesmann%2C+Louis&rft.date=2023-08-01&rft.issn=2377-3766&rft.eissn=2377-3766&rft.volume=8&rft.issue=8&rft.spage=4791&rft.epage=4798&rft_id=info:doi/10.1109%2FLRA.2023.3288383&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_LRA_2023_3288383
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon