Estimation, Control, and Planning for Aggressive Flight With a Small Quadrotor With a Single Camera and IMU

We address the state estimation, control, and planning for aggressive flight with a 150 cm diameter, 250 g quadrotor equipped only with a single camera and an inertial measurement unit (IMU). The use of smartphone grade hardware and the small scale provides an inexpensive and practical solution for...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE robotics and automation letters Vol. 2; no. 2; pp. 404 - 411
Main Authors:	Loianno, Giuseppe, Brunner, Chris, McGrath, Gary, Kumar, Vijay
Format:	Journal Article
Language:	English
Published:	IEEE 01.04.2017
Subjects:	Aerial robotics Cameras optimization and optimal control Planning Robot sensing systems sensor-based control State estimation Trajectory Vehicles
ISSN:	2377-3766, 2377-3766
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	We address the state estimation, control, and planning for aggressive flight with a 150 cm diameter, 250 g quadrotor equipped only with a single camera and an inertial measurement unit (IMU). The use of smartphone grade hardware and the small scale provides an inexpensive and practical solution for autonomous flight in indoor environments. The key contributions of this paper are: 1) robust state estimation and control using only a monocular camera and an IMU at speeds of 4.5 m/s, accelerations of over 1.5 g, roll and pitch angles of up to 90°, and angular rate of up to 800°/s without requiring any structure in the environment; 2) planning of dynamically feasible three-dimensional trajectories for slalom paths and flights through narrow windows; and 3) extensive experimental results showing aggressive flights through and around obstacles with large rotation angular excursions and accelerations.
AbstractList	We address the state estimation, control, and planning for aggressive flight with a 150 cm diameter, 250 g quadrotor equipped only with a single camera and an inertial measurement unit (IMU). The use of smartphone grade hardware and the small scale provides an inexpensive and practical solution for autonomous flight in indoor environments. The key contributions of this paper are: 1) robust state estimation and control using only a monocular camera and an IMU at speeds of 4.5 m/s, accelerations of over 1.5 g, roll and pitch angles of up to 90°, and angular rate of up to 800°/s without requiring any structure in the environment; 2) planning of dynamically feasible three-dimensional trajectories for slalom paths and flights through narrow windows; and 3) extensive experimental results showing aggressive flights through and around obstacles with large rotation angular excursions and accelerations.
Author	Loianno, Giuseppe Kumar, Vijay Brunner, Chris McGrath, Gary
Author_xml	– sequence: 1 givenname: Giuseppe surname: Loianno fullname: Loianno, Giuseppe email: loiannog@seas.upenn.edu organization: Gen. Robot., Autom., Sensing, & Perception Lab., Univ. of Pennsylvania, Philadelphia, PA, USA – sequence: 2 givenname: Chris surname: Brunner fullname: Brunner, Chris email: chris@qti.qualcomm.com organization: Qualcomm Technol., Inc., San Diego, CA, USA – sequence: 3 givenname: Gary surname: McGrath fullname: McGrath, Gary email: gmcgrath@qti.qualcomm.com organization: Qualcomm Technol., Inc., San Diego, CA, USA – sequence: 4 givenname: Vijay surname: Kumar fullname: Kumar, Vijay email: kumar@seas.upenn.edu organization: Gen. Robot., Autom., Sensing, & Perception Lab., Univ. of Pennsylvania, Philadelphia, PA, USA
BookMark	eNp9kFtLwzAUgINMcM69C77kB6zzJKEpfRxl08HE6_CxnKVpF81SSaPgvze7KOKDEMjhJN-5fKek51qnCTlnMGYM8svFw2TMgckxl0LwHI5In4ssS0QmZe9XfEKGXfcCACzlmcjTPnmddsFsMJjWjWjRuuBbO6LoKnpn0TnjGlq3nk6axuuuMx-azqxp1oE-m7CmSB83aC29f8fKtyF-_E5H0Gpa4EZ73JWb3yzPyHGNttPDwz0gy9n0qbhOFrdX82KySFQcPyQSldIqZbBKGSpcoaqwzrjOFQMRYy3zFUgdnyHlkENVM5HLmvHtAY1iQGBfV_m267yuyzcfd_SfJYNy66uMvsqtr_LgKyLyD6JM2FkJHo39D7zYg0Zr_dMnyyRnjIkvfxd6Ig
CODEN	IRALC6
CitedBy_id	crossref_primary_10_1109_LRA_2017_2699790 crossref_primary_10_1016_j_arcontrol_2023_03_009 crossref_primary_10_1109_LRA_2022_3154019 crossref_primary_10_1109_LRA_2018_2861888 crossref_primary_10_3390_s23249667 crossref_primary_10_1109_TSMC_2020_3037071 crossref_primary_10_1007_s10846_019_01031_z crossref_primary_10_1177_02783649221112446 crossref_primary_10_1109_TCST_2023_3283446 crossref_primary_10_1089_soro_2017_0120 crossref_primary_10_1002_rob_21956 crossref_primary_10_1109_LRA_2021_3052436 crossref_primary_10_1109_LRA_2024_3380923 crossref_primary_10_1016_j_ast_2021_106899 crossref_primary_10_1109_LRA_2018_2885575 crossref_primary_10_1109_TMECH_2019_2891776 crossref_primary_10_1109_TVT_2024_3382617 crossref_primary_10_1109_TRO_2022_3173711 crossref_primary_10_1109_LRA_2024_3484182 crossref_primary_10_1109_TRO_2024_3502508 crossref_primary_10_1109_TRO_2025_3548525 crossref_primary_10_1109_LRA_2020_2975723 crossref_primary_10_1109_TSMC_2019_2931812 crossref_primary_10_21595_jmeacs_2024_24602 crossref_primary_10_1109_TAES_2023_3321260 crossref_primary_10_1109_TCST_2019_2916089 crossref_primary_10_1007_s42405_021_00366_y crossref_primary_10_1016_j_conengprac_2023_105493 crossref_primary_10_1371_journal_pone_0264471 crossref_primary_10_3389_frobt_2019_00095 crossref_primary_10_3390_drones8030072 crossref_primary_10_1109_LRA_2018_2843445 crossref_primary_10_1016_j_neucom_2020_06_096 crossref_primary_10_1080_00207179_2021_1881165 crossref_primary_10_1109_JSSC_2018_2886342 crossref_primary_10_1109_LRA_2017_2775699 crossref_primary_10_1016_j_jfranklin_2020_04_056 crossref_primary_10_3390_drones7070471 crossref_primary_10_1109_TRO_2019_2942989 crossref_primary_10_1109_LRA_2019_2894220 crossref_primary_10_1016_j_isatra_2021_12_002 crossref_primary_10_1109_LRA_2025_3541376 crossref_primary_10_1016_j_robot_2020_103621 crossref_primary_10_1109_TVT_2023_3309554 crossref_primary_10_1109_LRA_2024_3505816 crossref_primary_10_1016_j_ifacol_2019_11_193 crossref_primary_10_1016_j_arcontrol_2019_08_004 crossref_primary_10_3390_electronics12071526 crossref_primary_10_1007_s10846_022_01574_8 crossref_primary_10_1109_LRA_2021_3051565 crossref_primary_10_1109_LRA_2021_3113976 crossref_primary_10_1109_LRA_2018_2795654 crossref_primary_10_1177_1756829319833685 crossref_primary_10_1146_annurev_control_060117_105149 crossref_primary_10_1109_LRA_2021_3067298 crossref_primary_10_1109_LRA_2018_2883408 crossref_primary_10_1109_TIE_2022_3144586 crossref_primary_10_1109_LRA_2021_3102324 crossref_primary_10_1177_0278364920921943 crossref_primary_10_1155_2018_4546094 crossref_primary_10_3390_s20247276 crossref_primary_10_1038_s41598_022_05798_4 crossref_primary_10_1109_LRA_2019_2918689 crossref_primary_10_1109_TIM_2019_2895495 crossref_primary_10_3390_s21103428 crossref_primary_10_1016_j_asoc_2022_109223 crossref_primary_10_1109_LRA_2023_3308490 crossref_primary_10_1007_s11432_018_9548_5 crossref_primary_10_1016_j_robot_2022_104292 crossref_primary_10_1002_rob_21887 crossref_primary_10_1007_s10846_021_01310_8 crossref_primary_10_1109_TIV_2022_3229723 crossref_primary_10_3390_s21186223 crossref_primary_10_1016_j_isatra_2025_06_034 crossref_primary_10_1109_LRA_2022_3192609 crossref_primary_10_3390_drones7020118 crossref_primary_10_1109_LRA_2021_3065286 crossref_primary_10_1109_TAES_2019_2937663 crossref_primary_10_1109_LRA_2022_3141661 crossref_primary_10_1126_scirobotics_abm5954 crossref_primary_10_1007_s10846_024_02214_z crossref_primary_10_1007_s40435_023_01357_5 crossref_primary_10_1007_s10514_019_09831_w crossref_primary_10_3390_rs13050972 crossref_primary_10_1016_j_trpro_2025_03_100 crossref_primary_10_1109_LRA_2017_2655145 crossref_primary_10_1109_LRA_2017_2778018 crossref_primary_10_1109_LRA_2025_3553785 crossref_primary_10_1109_LRA_2024_3455934 crossref_primary_10_1016_j_conengprac_2021_104827 crossref_primary_10_1109_LRA_2019_2895419 crossref_primary_10_1109_TRO_2021_3100142 crossref_primary_10_47134_jpo_v2i4_1952 crossref_primary_10_1109_TNNLS_2021_3107742 crossref_primary_10_1109_ACCESS_2024_3352131 crossref_primary_10_1109_LRA_2018_2800119 crossref_primary_10_1002_adem_202302214 crossref_primary_10_1109_TAES_2022_3143781 crossref_primary_10_1109_LCSYS_2021_3088406 crossref_primary_10_1016_j_mechatronics_2020_102455 crossref_primary_10_1109_LRA_2018_2868866 crossref_primary_10_1109_LRA_2025_3606806 crossref_primary_10_1002_rob_21743 crossref_primary_10_1109_LRA_2025_3606805 crossref_primary_10_1109_TRO_2023_3336320 crossref_primary_10_1007_s10846_019_01142_7 crossref_primary_10_3390_drones6040090 crossref_primary_10_1126_scirobotics_abl6259 crossref_primary_10_1109_ACCESS_2021_3052967 crossref_primary_10_1109_LRA_2020_2976308 crossref_primary_10_1109_TIV_2024_3391384 crossref_primary_10_1109_LRA_2017_2702198 crossref_primary_10_1109_LRA_2023_3295655 crossref_primary_10_2514_1_G004460 crossref_primary_10_1007_s11554_023_01318_3 crossref_primary_10_1109_TITS_2021_3125712 crossref_primary_10_3390_ijgi10100631 crossref_primary_10_1109_LRA_2024_3371254 crossref_primary_10_1109_TRO_2023_3339543 crossref_primary_10_1109_ACCESS_2020_2980016 crossref_primary_10_1109_TAES_2024_3379127 crossref_primary_10_1007_s10846_020_01252_7 crossref_primary_10_1007_s10846_021_01522_y crossref_primary_10_1109_TRO_2025_3567491 crossref_primary_10_2514_1_G006898 crossref_primary_10_1109_LRA_2022_3154024 crossref_primary_10_1109_TRO_2022_3141602 crossref_primary_10_1177_09544100221090690 crossref_primary_10_2514_1_G009084 crossref_primary_10_1109_TRO_2023_3238911 crossref_primary_10_1002_rob_22021 crossref_primary_10_1007_s11370_024_00582_x
Cites_doi	10.1109/IROS.2013.6696963 10.1109/ICRA.2015.7139420 10.1109/ICRA.2015.7139939 10.1002/rob.20412 10.1002/rob.21436 10.1177/0278364910382802 10.1109/IROS.2014.6942940 10.1109/MRA.2010.937855 10.1177/0278364910388963 10.1002/asjc.567 10.1177/0278364913509675 10.1109/LRA.2015.2506001 10.1109/TRO.2008.2003276 10.15607/RSS.2013.IX.032 10.1109/IROS.2015.7353530 10.1109/ROBOT.2010.5509627 10.1177/0278364911434236 10.1109/MRA.2012.2206473 10.1109/ACC.2012.6315568 10.1109/ICRA.2015.7139761 10.1109/ROBOT.2010.5509452 10.1109/ICRA.2011.5980409
ContentType	Journal Article
DBID	97E RIA RIE AAYXX CITATION
DOI	10.1109/LRA.2016.2633290
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef
DatabaseTitle	CrossRef
DatabaseTitleList
Database_xml	– sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2377-3766
EndPage	411
ExternalDocumentID	10_1109_LRA_2016_2633290 7762111
Genre	orig-research
GrantInformation_xml	– fundername: ARO grantid: W911NF-13-1-0350 funderid: 10.13039/100000183 – fundername: ARL grantid: W911NF-08-2-0004 funderid: 10.13039/100005423 – fundername: DARPA funderid: 10.13039/100000185 – fundername: ONR grantid: N00014-07-1-0829; N00014-14-1-0510 funderid: 10.13039/100000006 – fundername: NSF grantid: IIS-1426840; IIS-1138847 funderid: 10.13039/100000001 – fundername: MARCO and DARPA funderid: 10.13039/100007245 – fundername: Qualcomm Research – fundername: DARPA grantid: HR001151626; HR0011516850 funderid: 10.13039/100000185 – fundername: TerraSwarm
GroupedDBID	0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION
ID	FETCH-LOGICAL-c263t-6accec510b51acabacdaf72e9c103cdae69b06e0b5052090df1396f12f12f0ea3
IEDL.DBID	RIE
ISICitedReferencesCount	229
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000413736600005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2377-3766
IngestDate	Sat Nov 29 06:02:58 EST 2025 Tue Nov 18 21:40:01 EST 2025 Tue Aug 26 16:56:45 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	2
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c263t-6accec510b51acabacdaf72e9c103cdae69b06e0b5052090df1396f12f12f0ea3
PageCount	8
ParticipantIDs	crossref_primary_10_1109_LRA_2016_2633290 ieee_primary_7762111 crossref_citationtrail_10_1109_LRA_2016_2633290
PublicationCentury	2000
PublicationDate	2017-April 2017-4-00
PublicationDateYYYYMMDD	2017-04-01
PublicationDate_xml	– month: 04 year: 2017 text: 2017-April
PublicationDecade	2010
PublicationTitle	IEEE robotics and automation letters
PublicationTitleAbbrev	LRA
PublicationYear	2017
Publisher	IEEE
Publisher_xml	– name: IEEE
References	ref13 ref12 ref15 ref14 ref11 ref10 cacace (ref3) 2013 ref1 ref17 ozaslan (ref2) 2013 bullo (ref26) 2004; 49 ref16 ref19 ref18 ref24 ref23 ref25 ref20 ref22 ref21 ref7 abbeel (ref8) 2008 ref9 ref4 ref6 ref5
References_xml	– ident: ref15 doi: 10.1109/IROS.2013.6696963 – ident: ref21 doi: 10.1109/ICRA.2015.7139420 – ident: ref17 doi: 10.1109/ICRA.2015.7139939 – ident: ref16 doi: 10.1002/rob.20412 – start-page: 302 year: 2013 ident: ref3 publication-title: Aerial Service Vehicles for Industrial Inspection Task Decomposition and Plan Execution – ident: ref7 doi: 10.1002/rob.21436 – start-page: 123 year: 2013 ident: ref2 article-title: Inspection of penstocks and featureless tunnel-like environments using micro UAVs publication-title: Proc Int Conf Field Service Robot – ident: ref13 doi: 10.1177/0278364910382802 – ident: ref20 doi: 10.1109/IROS.2014.6942940 – ident: ref22 doi: 10.1109/MRA.2010.937855 – ident: ref12 doi: 10.1177/0278364910388963 – ident: ref23 doi: 10.1002/asjc.567 – year: 2008 ident: ref8 article-title: Apprenticeship learning and reinforcement learning with application to robotic control – volume: 49 year: 2004 ident: ref26 publication-title: Geometric Control of Mechanical Systems – ident: ref14 doi: 10.1177/0278364913509675 – ident: ref6 doi: 10.1109/LRA.2015.2506001 – ident: ref25 doi: 10.1109/TRO.2008.2003276 – ident: ref18 doi: 10.15607/RSS.2013.IX.032 – ident: ref19 doi: 10.1109/IROS.2015.7353530 – ident: ref9 doi: 10.1109/ROBOT.2010.5509627 – ident: ref11 doi: 10.1177/0278364911434236 – ident: ref1 doi: 10.1109/MRA.2012.2206473 – ident: ref5 doi: 10.1109/ACC.2012.6315568 – ident: ref4 doi: 10.1109/ICRA.2015.7139761 – ident: ref10 doi: 10.1109/ROBOT.2010.5509452 – ident: ref24 doi: 10.1109/ICRA.2011.5980409
SSID	ssj0001527395
Score	2.513486
Snippet	We address the state estimation, control, and planning for aggressive flight with a 150 cm diameter, 250 g quadrotor equipped only with a single camera and an...
SourceID	crossref ieee
SourceType	Enrichment Source Index Database Publisher
StartPage	404
SubjectTerms	Aerial robotics Cameras optimization and optimal control Planning Robot sensing systems sensor-based control State estimation Trajectory Vehicles
Title	Estimation, Control, and Planning for Aggressive Flight With a Small Quadrotor With a Single Camera and IMU
URI	https://ieeexplore.ieee.org/document/7762111
Volume	2
WOSCitedRecordID	wos000413736600005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE/IET Electronic Library (IEL) customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LT8IwGG-QeNCDLzTig_TgxYTBukdHj4RANAHiC-W2dF2rxDkMDo_-7X7txuRgTEx2aLp26fZb-72_D6EL4nFF7IhaHiPM8hwlLKDj1FLC9yT3uVDGev44DMbjznTKbiqoWcbCSCmN85ls6aax5cdzsdSqsnYAO5foQN4NaOWxWj_6FJ1JjPkrS6TN2sO7rnbdoi2Huq6jD901yrNWSsVQksHu_9awh3YKjhF3c4j3UUWmB2h7LY9gDb32YaPmMYhN3Mudz5uYpzFe1STCwJvi7rMRruF8w4NEC-X4aZa9YI7v33iS4NsljxdzkMHLbpiYSNzjWnFlHnc9mhyiyaD_0LuyiioKloAXzyzKhZACtl7kEy54xEXMVeBIJojtQltSFtlUwm3jEmPHCphCqoijL1ty9whV03kqjxFW3IfpcLa6jHt-RzEhnTggINUEHiXCraP26guHokgxritdJKERNWwWAiahxiQsMKmjy3LGe55e44-xNQ1HOa5A4uT37lO05WgKbJxszlA1WyzlOdoUn9nsY9FAG6OvfsP8Qd-DzcQo
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlR1NT8IwtCFooh78QiN-9uDFhMHabR09EgKBOIgfoNyW0rVKnMPg8PfbdmNyMCYmOzRd39Lt7fV9vwfANXKZRPaUWC5F1HKx5Jbi48SS3HMF8xiXxnv-FPjDYXMyoXclUCtyYYQQJvhM1PXQ-PKjOV9qU1nDV5SLdCLvhue62M6ytX4sKrqWGPVWvkibNoKHlg7eInVMHAfrY3eN96w1UzG8pLv3v13sg91cZoStDMkHoCSSQ7CzVkmwAt46ilSzLMQabGfh5zXIkgiuuhJBJZ3C1otRr9UJB7uxVsvh8yx9hQw-vrM4hvdLFi3mSgsvphVgLGCbadOVeVx_MD4C425n1O5ZeR8Fi6sXTy3COBdcEd_UQ4yzKeMRkz4WlCPbUWNB6NQmQt02QTF2JJVYSCTC-rIFc45BOZkn4gRAyTwFrk5XhzLXa0rKBY58pPQa3yWIO1XQWH3hkOdFxnWvizg0yoZNQ4WTUOMkzHFSBTcFxEdWYOOPtRWNjmJdjonT36evwFZvNAjCoD-8PQPbWPNjE3JzDsrpYikuwCb_Smefi0vzH30DO93GPg
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Estimation%2C+Control%2C+and+Planning+for+Aggressive+Flight+With+a+Small+Quadrotor+With+a+Single+Camera+and+IMU&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Loianno%2C+Giuseppe&rft.au=Brunner%2C+Chris&rft.au=McGrath%2C+Gary&rft.au=Kumar%2C+Vijay&rft.date=2017-04-01&rft.pub=IEEE&rft.eissn=2377-3766&rft.volume=2&rft.issue=2&rft.spage=404&rft.epage=411&rft_id=info:doi/10.1109%2FLRA.2016.2633290&rft.externalDocID=7762111
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon