FlightBench: Benchmarking Learning-Based Methods for Ego-Vision-Based Quadrotors Navigation

Ego-vision-based navigation in cluttered environments is crucial for mobile systems, particularly agile quadrotors. While learning-based methods have shown promise recently, head-to-head comparisons with cutting-edge optimization-based approaches are scarce, leaving open the question of where and to...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters Vol. 10; no. 7; pp. 6888 - 6895
Main Authors: Yu, Shu-Ang, Yu, Chao, Gao, Feng, Wu, Yi, Wang, Yu
Format: Journal Article
Language:English
Published: Piscataway IEEE 01.07.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Benchmark testing
Benchmarks
Criteria
deep learning methods
Flight characteristics
Forestry
Laser radar
Learning
Learning systems
Measurement
Navigation
Occlusion
Optimization
Performance measurement
Planning
Quadrotors
Rotary wing aircraft
Software tools for benchmarking and reproducibility
Three-dimensional displays
Training
vision-based navigation
ISSN:2377-3766, 2377-3766
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Ego-vision-based navigation in cluttered environments is crucial for mobile systems, particularly agile quadrotors. While learning-based methods have shown promise recently, head-to-head comparisons with cutting-edge optimization-based approaches are scarce, leaving open the question of where and to what extent they truly excel. In this letter, we introduce FlightBench, the first comprehensive benchmark that implements various learning-based methods for ego-vision-based navigation and evaluates them against mainstream optimization-based baselines using a broad set of performance metrics. More importantly, we develop a suite of criteria to assess scenario difficulty and design test cases that span different levels of difficulty based on these criteria. Our results show that while learning-based methods excel in high-speed flight and faster inference, they struggle with challenging scenarios like sharp corners or view occlusion. Analytical experiments validate the correlation between our difficulty criteria and flight performance.Moreover, we verify the trend in flight performance within real-world environments through full-pipeline and hardware-in-the-loop experiments. We hope this benchmark and these criteria will drive future advancements in learning-based navigation for ego-vision quadrotors.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2025.3573167