Learning high-speed flight in the wild
Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-...
Uloženo v:
| Vydáno v: | Science robotics Ročník 6; číslo 59; s. eabg5810 |
|---|---|
| Hlavní autoři: | , , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
United States
06.10.2021
|
| ISSN: | 2470-9476, 2470-9476 |
| On-line přístup: | Zjistit podrobnosti o přístupu |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. Although this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. The subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline. Here, we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and human-made environments at high speeds with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains, and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines. |
|---|---|
| AbstractList | Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. Although this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. The subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline. Here, we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and human-made environments at high speeds with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains, and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines. Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. Although this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. The subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline. Here, we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and human-made environments at high speeds with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains, and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines.Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. Although this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. The subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline. Here, we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and human-made environments at high speeds with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains, and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines. |
| Author | Ranftl, René Kaufmann, Elia Müller, Matthias Koltun, Vladlen Scaramuzza, Davide Loquercio, Antonio |
| Author_xml | – sequence: 1 givenname: Antonio orcidid: 0000-0002-8410-3933 surname: Loquercio fullname: Loquercio, Antonio organization: University of Zurich, Zürich, Switzerland – sequence: 2 givenname: Elia orcidid: 0000-0001-6094-5901 surname: Kaufmann fullname: Kaufmann, Elia organization: University of Zurich, Zürich, Switzerland – sequence: 3 givenname: René orcidid: 0000-0003-4158-2759 surname: Ranftl fullname: Ranftl, René organization: Intel, Munich, Germany – sequence: 4 givenname: Matthias orcidid: 0000-0001-5249-8734 surname: Müller fullname: Müller, Matthias organization: Intel, Munich, Germany – sequence: 5 givenname: Vladlen orcidid: 0000-0003-0858-0970 surname: Koltun fullname: Koltun, Vladlen organization: Intel, Santa Clara, CA, USA – sequence: 6 givenname: Davide orcidid: 0000-0002-3831-6778 surname: Scaramuzza fullname: Scaramuzza, Davide organization: University of Zurich, Zürich, Switzerland |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/34613820$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNj81KxDAYRYOMOOM4TyBIV-KmY74kTdOlDOMPFNzouiRf0jbSP5sW8e0tOIKrexaHC-eSrLq-c4RcA90DMHkf0I-96SePYa9NlSigZ2TDRErjTKRy9Y_XZBfCB6UUUsmlYBdkzYUErhjdkNvc6bHzXRXVvqrjMDhno7JZeIp8F021i758Y6_Ieamb4Han3ZL3x-Pb4TnOX59eDg95jJwmU4wgrXIlYJmWiMZxyIDqREuDNmVUZEKhRSuUTrnSGVLBAUSJFBhfqgzbkrvf32HsP2cXpqL1AV3T6M71cyhYopYOqRK5qDcndTats8Uw-laP38VfG_sBlvRUkw |
| CitedBy_id | crossref_primary_10_1007_s10846_024_02091_6 crossref_primary_10_1016_j_ifacol_2022_07_595 crossref_primary_10_1109_JIOT_2024_3367451 crossref_primary_10_1109_LRA_2023_3281261 crossref_primary_10_1109_TAI_2024_3366871 crossref_primary_10_1016_j_compag_2025_110802 crossref_primary_10_1126_scirobotics_abl8419 crossref_primary_10_1109_LRA_2022_3207545 crossref_primary_10_1016_j_engappai_2025_112090 crossref_primary_10_1038_s42256_025_01050_6 crossref_primary_10_1109_LRA_2022_3176449 crossref_primary_10_1126_scirobotics_ado6187 crossref_primary_10_1109_LRA_2024_3522778 crossref_primary_10_1126_scirobotics_ads5033 crossref_primary_10_1109_TITS_2024_3375893 crossref_primary_10_1109_TIV_2022_3163315 crossref_primary_10_1002_rob_22523 crossref_primary_10_3390_drones7120690 crossref_primary_10_7746_jkros_2024_19_1_117 crossref_primary_10_3390_drones7030192 crossref_primary_10_1109_JSEN_2025_3596433 crossref_primary_10_1109_TNNLS_2024_3525264 crossref_primary_10_1126_scirobotics_adf6991 crossref_primary_10_1109_TRO_2024_3400838 crossref_primary_10_1109_LRA_2022_3181755 crossref_primary_10_1109_LRA_2025_3560842 crossref_primary_10_1371_journal_pone_0264471 crossref_primary_10_1177_02783649231218720 crossref_primary_10_3390_drones8030072 crossref_primary_10_1016_j_ast_2024_109812 crossref_primary_10_3390_s21227436 crossref_primary_10_1109_TIV_2024_3352613 crossref_primary_10_3390_drones7110651 crossref_primary_10_3390_s22249830 crossref_primary_10_1016_j_eswa_2022_116742 crossref_primary_10_1109_TIM_2021_3135544 crossref_primary_10_1007_s10846_023_02042_7 crossref_primary_10_1177_01423312241228886 crossref_primary_10_3390_drones7120704 crossref_primary_10_1109_LRA_2025_3595077 crossref_primary_10_1038_s41467_024_46967_5 crossref_primary_10_1109_TCDS_2024_3519319 crossref_primary_10_1109_LRA_2024_3426374 crossref_primary_10_1126_scirobotics_abm6597 crossref_primary_10_1016_j_ifacol_2023_01_146 crossref_primary_10_1038_s41467_024_54451_3 crossref_primary_10_3389_frobt_2023_1285412 crossref_primary_10_3390_drones7100609 crossref_primary_10_1016_j_eja_2024_127477 crossref_primary_10_1109_TRO_2023_3315710 crossref_primary_10_1109_LRA_2024_3397540 crossref_primary_10_1109_TCST_2023_3297744 crossref_primary_10_1109_LRA_2022_3194682 crossref_primary_10_1007_s41315_025_00457_z crossref_primary_10_1109_LRA_2024_3357399 crossref_primary_10_1038_s41598_023_36775_0 crossref_primary_10_1002_rob_22460 crossref_primary_10_1109_LRA_2025_3580318 crossref_primary_10_3390_drones8120782 crossref_primary_10_1109_LRA_2024_3371288 crossref_primary_10_3390_aerospace10050418 crossref_primary_10_1002_aisy_202400916 crossref_primary_10_1109_LRA_2025_3562004 crossref_primary_10_1109_TRO_2024_3508140 crossref_primary_10_1109_LRA_2023_3314350 crossref_primary_10_1109_LRA_2025_3606383 crossref_primary_10_1177_02783649231169803 crossref_primary_10_1109_LRA_2022_3150497 crossref_primary_10_3390_s23167206 crossref_primary_10_1109_OJVT_2024_3402129 crossref_primary_10_1109_LRA_2025_3592101 crossref_primary_10_1016_j_knosys_2024_112124 crossref_primary_10_1109_LRA_2025_3548504 crossref_primary_10_1049_ipr2_13079 crossref_primary_10_1109_TVT_2023_3278097 crossref_primary_10_1016_j_trpro_2025_03_100 crossref_primary_10_1109_TRO_2024_3428433 crossref_primary_10_1109_LRA_2024_3399589 crossref_primary_10_1038_s41586_023_06419_4 crossref_primary_10_1007_s11071_024_10029_8 crossref_primary_10_1109_LRA_2025_3553785 crossref_primary_10_1002_aisy_202200113 crossref_primary_10_1109_TRO_2024_3522187 crossref_primary_10_1038_s42256_025_01048_0 crossref_primary_10_1109_TNNLS_2025_3564184 crossref_primary_10_1109_TRO_2023_3334630 crossref_primary_10_1109_ACCESS_2024_3421661 crossref_primary_10_1109_TASE_2024_3432405 crossref_primary_10_1109_LRA_2024_3518078 crossref_primary_10_1109_TIE_2025_3558030 crossref_primary_10_1109_LRA_2025_3547306 crossref_primary_10_1109_LRA_2024_3355632 crossref_primary_10_1016_j_est_2023_107677 crossref_primary_10_1146_annurev_control_030323_022510 crossref_primary_10_1109_LRA_2025_3606806 crossref_primary_10_1109_LRA_2025_3606805 crossref_primary_10_1016_j_birob_2025_100253 crossref_primary_10_1088_2634_4386_ac7ee0 crossref_primary_10_1126_scirobotics_abl6259 crossref_primary_10_1109_TNNLS_2023_3331370 crossref_primary_10_1109_ACCESS_2024_3440183 crossref_primary_10_1109_JIOT_2024_3363036 crossref_primary_10_1109_LRA_2025_3573167 crossref_primary_10_1109_TAES_2025_3535856 crossref_primary_10_1007_s10462_025_11348_x crossref_primary_10_1177_02783649251353181 crossref_primary_10_3390_robotics11050094 crossref_primary_10_1109_LRA_2022_3146897 crossref_primary_10_1126_scirobotics_adi9641 crossref_primary_10_1109_ACCESS_2023_3330230 crossref_primary_10_1109_TASE_2025_3606896 crossref_primary_10_1109_JIOT_2024_3450967 crossref_primary_10_1109_LRA_2023_3295655 crossref_primary_10_1016_j_robot_2023_104565 crossref_primary_10_1109_TPAMI_2024_3395484 crossref_primary_10_3390_s24196258 crossref_primary_10_1109_TCCN_2024_3358545 crossref_primary_10_1109_TRO_2025_3526102 crossref_primary_10_1109_LRA_2023_3235678 crossref_primary_10_1145_3703453 crossref_primary_10_3390_drones7020144 crossref_primary_10_1109_LRA_2024_3386053 crossref_primary_10_1109_TIE_2023_3323750 crossref_primary_10_1126_scirobotics_add5139 |
| ContentType | Journal Article |
| DBID | NPM 7X8 |
| DOI | 10.1126/scirobotics.abg5810 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | PubMed MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2470-9476 |
| ExternalDocumentID | 34613820 |
| Genre | Research Support, Non-U.S. Gov't Journal Article |
| GroupedDBID | 0R~ ABJNI ACGFS AJGZS ALMA_UNASSIGNED_HOLDINGS ARCSS BKF EBS NPM O9- SJN 7X8 |
| ID | FETCH-LOGICAL-c305t-c16d8ef1cf7fccbe31910a5a6bcd7204948cdcd48a738a9c043114fc0123126b2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 209 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000705912200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2470-9476 |
| IngestDate | Sun Nov 09 11:02:39 EST 2025 Wed Feb 19 02:27:49 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 59 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c305t-c16d8ef1cf7fccbe31910a5a6bcd7204948cdcd48a738a9c043114fc0123126b2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0002-8410-3933 0000-0002-3831-6778 0000-0001-5249-8734 0000-0003-0858-0970 0000-0001-6094-5901 0000-0003-4158-2759 |
| PMID | 34613820 |
| PQID | 2580016856 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_2580016856 pubmed_primary_34613820 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-10-06 |
| PublicationDateYYYYMMDD | 2021-10-06 |
| PublicationDate_xml | – month: 10 year: 2021 text: 2021-10-06 day: 06 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Science robotics |
| PublicationTitleAlternate | Sci Robot |
| PublicationYear | 2021 |
| SSID | ssj0001763642 |
| Score | 2.6176298 |
| Snippet | Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | eabg5810 |
| Title | Learning high-speed flight in the wild |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/34613820 https://www.proquest.com/docview/2580016856 |
| Volume | 6 |
| WOSCitedRecordID | wos000705912200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEB7UetCD70d9sYJ4i-4jm01OImLxYulBobeSzaMIslu71d_vzDalXgTBS24LYXYm82Vm8n0AV7GILRciZjopFeOYAJj0XjGrlM-z3BlteSs2UfT7cjhUg1Bwa8JY5eJMbA9qWxuqkd-muSR4InNxN_lgpBpF3dUgobEKnQyhDI10FUO5rLFg8IhWPyflRcwUL0QgHgrvZqZ1WRMb8o0ux7lM4t9hZptuetv_3egObAWgGd3PPWMXVly1B5s_6Af34TqQq44jIi1mzQQzWeTf6boevVURQsMIkbQ9gNfe48vDEwu6Ccxg9M6YSYSVzifGF96Y0mGUJbHOtSiNJU0axaWxxnKpi0xqZYhfJ-HeELxCw5TpIaxVdeWOIbKCnprKLBOGGr9OE5-YwkujUKmzievC5cIII_RLajboytWfzWhphi4czS05mswJNEYZ-kSG0OPkD1-fwkZKYyTUsxdn0PEYle4c1s3X7K2ZXrQ_HNf-4Pkbx92z6w |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Learning+high-speed+flight+in+the+wild&rft.jtitle=Science+robotics&rft.au=Loquercio%2C+Antonio&rft.au=Kaufmann%2C+Elia&rft.au=Ranftl%2C+Ren%C3%A9&rft.au=M%C3%BCller%2C+Matthias&rft.date=2021-10-06&rft.eissn=2470-9476&rft.volume=6&rft.issue=59&rft.spage=eabg5810&rft_id=info:doi/10.1126%2Fscirobotics.abg5810&rft_id=info%3Apmid%2F34613820&rft_id=info%3Apmid%2F34613820&rft.externalDocID=34613820 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2470-9476&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2470-9476&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2470-9476&client=summon |