Real-time Instance-Aware Segmentation and Semantic Mapping on Edge Devices

Perceiving the environment semantically in real-time is challenging for unmanned aerial vehicles (UAVs) with limited computational resources. In this paper, a real-time instance-aware segmentation and semantic mapping method on small edge devices is proposed. Taking RGB-D image as input, the present...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on instrumentation and measurement Jg. 72; S. 1
Hauptverfasser: Lu, Junjie, Tian, Bailing, Shen, Hongming, Zhang, Xuewei
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 01.01.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
autonomous exploration
Feature extraction
Image edge detection
Image segmentation
instance segmentation
Mapping
Object recognition
Real time
Real-time systems
Semantic mapping
Semantics
Simultaneous localization and mapping
Statistical analysis
Three-dimensional displays
Unmanned aerial vehicles
ISSN:0018-9456, 1557-9662
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Perceiving the environment semantically in real-time is challenging for unmanned aerial vehicles (UAVs) with limited computational resources. In this paper, a real-time instance-aware segmentation and semantic mapping method on small edge devices is proposed. Taking RGB-D image as input, the presented instance segmentation pipeline is able to run at the speed of 38 FPS on AGX Xavier. To achieve this, we take a lightweight object detection model as backbone and reformulate the mask generation problem as threshold regression in depth by a novel designed truncation network. After that, a probability grid map is constructed to integrate the categories of voxels and object-level entities. Objects parameterized by pose, extent, category, and point cloud are tracked and fused across frames by data association. Finally, autonomous exploration experiments of UAV are conducted to demonstrate the effectiveness of the proposed method in both simulation and real-world.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9456
1557-9662
DOI:10.1109/TIM.2022.3224512