Learning Occluded Branch Depth Maps in Forest Environments Using RGB-D Images

Covering over a third of all terrestrial land area, forests are crucial environments; as ecosystems, for farming, and for human leisure. However, they are challenging to access for environmental monitoring, for agricultural uses, and for search and rescue applications. To enter, aerial robots need t...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters Vol. 9; no. 3; pp. 2439 - 2446
Main Authors: Geckeler, Christian, Aucone, Emanuele, Schnider, Yannick, Simeon, Andri, von Bassewitz, Jan-Philipp, Zhu, Yunying, Mintchev, Stefano
Format: Journal Article
Language:English
Published: Piscataway The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 01.03.2024
Subjects:
Data augmentation
Datasets
Environmental monitoring
Foliage
Forests
Image reconstruction
Pixels
Rainforests
Robots
Trees
ISSN:2377-3766, 2377-3766
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Covering over a third of all terrestrial land area, forests are crucial environments; as ecosystems, for farming, and for human leisure. However, they are challenging to access for environmental monitoring, for agricultural uses, and for search and rescue applications. To enter, aerial robots need to fly through dense vegetation, where foliage can be pushed aside, but occluded branches pose critical obstacles. Therefore, we propose pixel-wise depth regression of occluded branches using three different U-Net inspired architectures. Given RGB-D input of trees with partially occluded branches, the models estimate depth values of only the wooden parts of the tree. A large photorealistic simulation dataset comprising around 44 K images of nine different tree species is generated, on which the models are trained. Extensive evaluation and analysis of the models on this dataset is shown. To improve network generalization to real-world data, different data augmentation and transformation techniques are performed. The approaches are then also successfully demonstrated on real-world data of broadleaf trees from Swiss temperate forests and a tropical Masoala Rainforest. This work showcases the previously unexplored task of frame-by-frame pixel-based occluded branch depth reconstruction to facilitate robot traversal of forest environments.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2024.3355632