Spatio-Temporal Consistent Semantic Mapping for Robotics Fruit Growth Monitoring

Automatic fruit growth monitoring plays a vital role in advancing precision agriculture. Tracking the evolution of fruits over time is essential to monitor their development and optimize production. The ability to recognize fruits over periods of time, even with drastic scene changes, is a required...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE robotics and automation letters Jg. 10; H. 9; S. 9470 - 9477
Hauptverfasser: Lobefaro, Luca, Sodano, Matteo, Fusaro, Daniel, Magistri, Federico, Malladi, Meher V. R., Guadagnino, Tiziano, Pretto, Alberto, Stachniss, Cyrill
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Piscataway IEEE 01.09.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Annotations
Cameras
Computational modeling
Fruits
Image segmentation
Instance segmentation
Mapping
Monitoring
Point cloud compression
Robotics
robotics and automation in agriculture and forestry
Robots
Semantics
Solid modeling
Three dimensional models
Three-dimensional displays
Tracking
ISSN:2377-3766, 2377-3766
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Automatic fruit growth monitoring plays a vital role in advancing precision agriculture. Tracking the evolution of fruits over time is essential to monitor their development and optimize production. The ability to recognize fruits over periods of time, even with drastic scene changes, is a required capability of agricultural robots. This letter presents a system that allows long-term fruit tracking in 3D data. It generates instance-segmented 3D representations of plants at various growth stages over time, utilizing only consumer-grade RGB-D cameras installed on a mobile robot. Our approach first performs instance segmentation on each image in a sequence. Then, by exploiting geometric information and depth maps, we track the same instances throughout the sequence. We produce a 3D point cloud containing instances, exploiting odometry information and 3D semantic mapping. Once our robot performs a new recording at a different plant growth stage, it associates each fruit with the previously built 3D cloud and update the model. We validate the system in a real-world glasshouse environment in Bonn, Germany. Experimental results demonstrate that our system outperforms existing baselines even though it relies only on annotated images and operates at frame-rate, allowing the deployment on a real robot.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2025.3594985