Contrastive 3D Shape Completion and Reconstruction for Agricultural Robots Using RGB-D Frames

Monitoring plants and fruits is important in modern agriculture, with applications ranging from high-throughput phenotyping to autonomous harvesting. Obtaining highly accurate 3D measurements under real agricultural conditions is a challenging task. In this letter, we address the problem of estimati...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE robotics and automation letters Jg. 7; H. 4; S. 10120 - 10127
Hauptverfasser: Magistri, Federico, Marks, Elias, Nagulavancha, Sumanth, Vizzo, Ignacio, Labe, Thomas, Behley, Jens, Halstead, Michael, McCool, Chris, Stachniss, Cyrill
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Piscataway IEEE 01.10.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Coders
Crops
Deep learning for visual perception
Fruits
Greenhouses
Harvesting
RGB-D perception
Robot sensing systems
robotics and automation in agriculture and forestry
Robots
Shape
Task analysis
Three-dimensional displays
ISSN:2377-3766, 2377-3766
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Monitoring plants and fruits is important in modern agriculture, with applications ranging from high-throughput phenotyping to autonomous harvesting. Obtaining highly accurate 3D measurements under real agricultural conditions is a challenging task. In this letter, we address the problem of estimating the 3D shape of fruits when only a partial view is available. We propose a pipeline that exploits high-resolution 3D data in the learning phase but only requires a single RGB-D frame to predict the 3D shape of a complete fruit during operation. To achieve this, we first learn a latent space of potential fruit appearances that we can decode into an SDF volume. With the pretrained, frozen decoder, we subsequently learn an encoder that can produce meaningful latent vectors from a single RGB-D frame. The experiments presented in this letter suggest that our approach can predict the 3D shape of whole fruits online, needing only 4 ms for inference. We evaluate our approach in controlled environments and illustrate its deployment in greenhouses without modifications.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2022.3193239