Small object detection in remote sensing images based on attention mechanism and multi-scale feature fusion

Due to the influence of dense distribution of detection objects and complex background, there are many small objects, which are difficult to detect in remote sensing images. In order to solve the difficult problem of small object detection in remote sensing images, we propose an object detection alg...

Full description

Saved in:

Bibliographic Details
Published in:	International journal of remote sensing Vol. 43; no. 9; pp. 3280 - 3297
Main Authors:	Zhang, Li-guo, Wang, Lei, Jin, Mei, Geng, Xing-shuo, Shen, Qian
Format:	Journal Article
Language:	English
Published:	London Taylor & Francis 03.05.2022 Taylor & Francis Ltd
Subjects:	Algorithms Backbone Clustering Convolution Cot module CotYOLO-v3 object detection algorithm data collection Datasets Detection feature fusion Object recognition prediction Redesign Remote sensing Sampling methods Small object detection
ISSN:	0143-1161, 1366-5901, 1366-5901
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Due to the influence of dense distribution of detection objects and complex background, there are many small objects, which are difficult to detect in remote sensing images. In order to solve the difficult problem of small object detection in remote sensing images, we propose an object detection algorithm named CotYOLO-v3 in this paper. First, we redesign the residual blocks in the backbone Darknet-53, and we replace it with Contextual Transformer (Cot) blocks with contextual information in the backbone Darknet-53 to extract contextual information for small objects and enhance visual representation; Second, we introduce the shallow information with attention mechanism before the feature fusion of YOLO-v3 to reduce the influence of background interference factors and improve the expression ability of the network. Then, we optimize the feature fusion process, we replace the up-sampling method with sub-pixel convolution, and we replace the first convolution layer of the prediction branch with a residual block. Finally, we use K-Medians clustering algorithm to regenerate the anchors suitable for the remote sensing image datasets. In this paper, we set up a comparative experiment of CotYOLO-v3 and commonly used object detection algorithms to detect small objects in DIOR datasets. The experimental results show that, compared with other commonly used object detection algorithms, CotYOLO-v3 object detection algorithm has obvious advantages in detecting small objects in remote sensing images. Compared with the original object detection algorithm YOLO-v3, the mean Average Precision (mAP) of CotYOLO-v3 improved by 5.07%.
AbstractList	Due to the influence of dense distribution of detection objects and complex background, there are many small objects, which are difficult to detect in remote sensing images. In order to solve the difficult problem of small object detection in remote sensing images, we propose an object detection algorithm named CotYOLO-v3 in this paper. First, we redesign the residual blocks in the backbone Darknet-53, and we replace it with Contextual Transformer (Cot) blocks with contextual information in the backbone Darknet-53 to extract contextual information for small objects and enhance visual representation; Second, we introduce the shallow information with attention mechanism before the feature fusion of YOLO-v3 to reduce the influence of background interference factors and improve the expression ability of the network. Then, we optimize the feature fusion process, we replace the up-sampling method with sub-pixel convolution, and we replace the first convolution layer of the prediction branch with a residual block. Finally, we use K-Medians clustering algorithm to regenerate the anchors suitable for the remote sensing image datasets. In this paper, we set up a comparative experiment of CotYOLO-v3 and commonly used object detection algorithms to detect small objects in DIOR datasets. The experimental results show that, compared with other commonly used object detection algorithms, CotYOLO-v3 object detection algorithm has obvious advantages in detecting small objects in remote sensing images. Compared with the original object detection algorithm YOLO-v3, the mean Average Precision (mAP) of CotYOLO-v3 improved by 5.07%.
Author	Jin, Mei Shen, Qian Wang, Lei Zhang, Li-guo Geng, Xing-shuo
Author_xml	– sequence: 1 givenname: Li-guo surname: Zhang fullname: Zhang, Li-guo organization: Yanshan University – sequence: 2 givenname: Lei surname: Wang fullname: Wang, Lei email: ysuwanglei@163.com organization: Yanshan University – sequence: 3 givenname: Mei surname: Jin fullname: Jin, Mei organization: Yanshan University – sequence: 4 givenname: Xing-shuo surname: Geng fullname: Geng, Xing-shuo organization: Yanshan University – sequence: 5 givenname: Qian surname: Shen fullname: Shen, Qian organization: Yanshan University
BookMark	eNqFkU9rFjEQxoNU8G31IwgBL162TrLZbBYvSvEfFDyo55DNTmpes0lNski_vVnfeulBDzMPCb9nGOY5J2cxRSTkOYNLBgpeARM9Y5JdcuC8NTUN_fSIHFgvZTdMwM7IYWe6HXpCzks5AoAch_FAfnxZTQg0zUe0lS5Ym_gUqY8045oq0oKx-HhD_WpusNDZFFxoI0ytGP-wK9rvJvqyUhMXum6h-q5YE5A6NHXLTbfSwKfksTOh4LN7vSDf3r_7evWxu_784dPV2-vO9iOrHZ-Fss3ppGIgQQhlrOAA3AkxziDkhG5pxcDCbNtTzO1_cBNzuAxc9Rfk5WnubU4_NyxVr75YDMFETFvRfGSKK-CKN_TFA_SYthzbdprLiUmYBBsbNZwom1MpGZ2-ze0c-U4z0HsE-m8Eeo9A30fQfK8f-KyvZr9ZzcaH_7rfnNw-upRX8yvlsOhq7kLKLptofdH9v0f8BlKAoY4
CitedBy_id	crossref_primary_10_3390_s24165392 crossref_primary_10_1109_ACCESS_2024_3414426 crossref_primary_10_1109_JSTARS_2024_3392635 crossref_primary_10_3390_e25020298 crossref_primary_10_1109_LGRS_2023_3344937 crossref_primary_10_1016_j_ins_2023_120074 crossref_primary_10_14358_PERS_23_00004R3 crossref_primary_10_3390_rs15235575 crossref_primary_10_1117_1_JEI_32_6_063016 crossref_primary_10_1016_j_patrec_2024_04_002 crossref_primary_10_1016_j_eswa_2025_129435 crossref_primary_10_1007_s11554_025_01710_1 crossref_primary_10_1007_s00371_023_02886_y crossref_primary_10_3390_app13095231
Cites_doi	10.48550/arXiv.1912.06319 10.1109/TPAMI.2016.2577031 10.48550/arXiv.1804.02767 10.1007/978-1-4842-2766-4 10.48550/arXiv.1701.06659 10.1109/CVPR.2016.207 10.11809/bqzbgcxb2021.07.029 10.1109/CVPR.2016.91 10.1080/01431161.2022.2038396 10.1109/CVPR.2017.211 10.5121/csit.2019.91713 10.1007/978-3-319-46448-0_2 10.1007/978-3-319-46493-0_22 10.48550/arXiv.2003.07021 10.1007/978-3-642-35289-8_30 10.1109/CVPR.2014.81 10.1145/3422622 10.1109/LSP.2016.2603342 10.48550/arXiv.2107.12292 10.1109/CVPR.2017.106 10.1109/ICCV.2015.169 10.1016/j.isprsjprs.2019.11.023 10.1109/CVPR.2017.690 10.1109/TPAMI.2019.2913372 10.1080/21642583.2021.1901156
ContentType	Journal Article
Copyright	2022 Informa UK Limited, trading as Taylor & Francis Group 2022 2022 Informa UK Limited, trading as Taylor & Francis Group
Copyright_xml	– notice: 2022 Informa UK Limited, trading as Taylor & Francis Group 2022 – notice: 2022 Informa UK Limited, trading as Taylor & Francis Group
DBID	AAYXX CITATION 7TG 7TN 8FD F1W FR3 H8D H96 KL. KR7 L.G L7M 7S9 L.6
DOI	10.1080/01431161.2022.2089539
DatabaseName	CrossRef Meteorological & Geoastrophysical Abstracts Oceanic Abstracts Technology Research Database ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Aerospace Database Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources Meteorological & Geoastrophysical Abstracts - Academic Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Advanced Technologies Database with Aerospace AGRICOLA AGRICOLA - Academic
DatabaseTitle	CrossRef Aerospace Database Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Meteorological & Geoastrophysical Abstracts Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources Oceanic Abstracts Technology Research Database ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Meteorological & Geoastrophysical Abstracts - Academic AGRICOLA AGRICOLA - Academic
DatabaseTitleList	Aerospace Database AGRICOLA
DeliveryMethod	fulltext_linktorsrc
Discipline	Geography
EISSN	1366-5901
EndPage	3297
ExternalDocumentID	10_1080_01431161_2022_2089539 2089539
Genre	Research Article
GroupedDBID	-~X .7F .DC .QJ 0BK 0R~ 29J 30N 4.4 5GY 5VS AAENE AAGDL AAHBH AAHIA AAJMT AALDU AAMIU AAPUL AAQRR ABCCY ABFIM ABHAV ABJNI ABLIJ ABLJU ABPAQ ABPEM ABRLO ABUFD ABXUL ABXYU ACGEJ ACGFS ACIWK ACTIO ADCVX ADGTB ADXPE AEISY AENEX AEOZL AEPSL AEXLP AEYOC AFKVX AFRVT AGDLA AGMYJ AHDZW AIJEM AIYEW AJWEG AKBVH AKOOK ALMA_UNASSIGNED_HOLDINGS ALQZU AQRUH AQTUD AVBZW AWYRJ BLEHA CCCUG CE4 CS3 DGEBU DKSSO DU5 EBS E~A E~B F5P H13 HF~ IPNFZ J.P KYCEM LJTGL M4Z P2P RIG RNANH ROSJB RTWRZ S-T SNACF TASJS TBQAZ TDBHL TEN TFL TFT TFW TN5 TNC TQWBC TTHFI TUROJ TWF UPT UT5 UU3 ZGOLN ~02 ~S~ AAYXX CITATION 7TG 7TN 8FD F1W FR3 H8D H96 KL. KR7 L.G L7M 7S9 L.6
ID	FETCH-LOGICAL-c371t-2b48cfeaf681060448ac42002f447b0469efd9ef10c0bc4694b47b5f91fed5283
IEDL.DBID	TFW
ISICitedReferencesCount	14
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000825130200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0143-1161 1366-5901
IngestDate	Fri Sep 05 17:31:33 EDT 2025 Wed Aug 13 06:17:25 EDT 2025 Sat Nov 29 06:13:47 EST 2025 Tue Nov 18 21:42:17 EST 2025 Mon Oct 20 23:47:24 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	9
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c371t-2b48cfeaf681060448ac42002f447b0469efd9ef10c0bc4694b47b5f91fed5283
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23
PQID	2691609417
PQPubID	2045515
PageCount	18
ParticipantIDs	informaworld_taylorfrancis_310_1080_01431161_2022_2089539 crossref_primary_10_1080_01431161_2022_2089539 proquest_journals_2691609417 proquest_miscellaneous_2718280282 crossref_citationtrail_10_1080_01431161_2022_2089539
PublicationCentury	2000
PublicationDate	2022-05-03
PublicationDateYYYYMMDD	2022-05-03
PublicationDate_xml	– month: 05 year: 2022 text: 2022-05-03 day: 03
PublicationDecade	2020
PublicationPlace	London
PublicationPlace_xml	– name: London
PublicationTitle	International journal of remote sensing
PublicationYear	2022
Publisher	Taylor & Francis Taylor & Francis Ltd
Publisher_xml	– name: Taylor & Francis – name: Taylor & Francis Ltd
References	cit0011 cit0012 cit0010 cit0019 cit0017 cit0018 cit0015 cit0016 cit0013 cit0014 cit0022 cit0001 cit0023 cit0020 cit0021 Fang Q. (cit0006) 2019; 036 cit0008 cit0009 cit0028 cit0007 cit0029 cit0004 cit0026 cit0005 cit0027 cit0002 cit0024 cit0003 cit0025
References_xml	– ident: cit0018 doi: 10.48550/arXiv.1912.06319 – ident: cit0025 doi: 10.1109/TPAMI.2016.2577031 – ident: cit0024 doi: 10.48550/arXiv.1804.02767 – ident: cit0013 doi: 10.1007/978-1-4842-2766-4 – ident: cit0007 doi: 10.48550/arXiv.1701.06659 – ident: cit0026 doi: 10.1109/CVPR.2016.207 – ident: cit0011 doi: 10.11809/bqzbgcxb2021.07.029 – ident: cit0022 doi: 10.1109/CVPR.2016.91 – ident: cit0015 doi: 10.1080/01431161.2022.2038396 – ident: cit0016 doi: 10.1109/CVPR.2017.211 – ident: cit0014 doi: 10.5121/csit.2019.91713 – ident: cit0021 – ident: cit0020 doi: 10.1007/978-3-319-46448-0_2 – ident: cit0001 doi: 10.1007/978-3-319-46493-0_22 – ident: cit0004 doi: 10.48550/arXiv.2003.07021 – volume: 036 start-page: 21 issue: 005 year: 2019 ident: cit0006 publication-title: Shanghai Aerospace – ident: cit0002 doi: 10.1007/978-3-642-35289-8_30 – ident: cit0008 doi: 10.1109/CVPR.2014.81 – ident: cit0010 doi: 10.1145/3422622 – ident: cit0029 doi: 10.1109/LSP.2016.2603342 – ident: cit0028 doi: 10.48550/arXiv.2107.12292 – ident: cit0019 doi: 10.1109/CVPR.2017.106 – ident: cit0009 doi: 10.1109/ICCV.2015.169 – ident: cit0003 – ident: cit0017 doi: 10.1016/j.isprsjprs.2019.11.023 – ident: cit0023 doi: 10.1109/CVPR.2017.690 – ident: cit0005 – ident: cit0012 doi: 10.1109/TPAMI.2019.2913372 – ident: cit0027 doi: 10.1080/21642583.2021.1901156
SSID	ssj0006757
Score	2.4466124
Snippet	Due to the influence of dense distribution of detection objects and complex background, there are many small objects, which are difficult to detect in remote...
SourceID	proquest crossref informaworld
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	3280
SubjectTerms	Algorithms Backbone Clustering Convolution Cot module CotYOLO-v3 object detection algorithm data collection Datasets Detection feature fusion Object recognition prediction Redesign Remote sensing Sampling methods Small object detection
Title	Small object detection in remote sensing images based on attention mechanism and multi-scale feature fusion
URI	https://www.tandfonline.com/doi/abs/10.1080/01431161.2022.2089539 https://www.proquest.com/docview/2691609417 https://www.proquest.com/docview/2718280282
Volume	43
WOSCitedRecordID	wos000825130200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAWR databaseName: Taylor & Francis Journals Complete customDbUrl: eissn: 1366-5901 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0006757 issn: 0143-1161 databaseCode: TFW dateStart: 19800101 isFulltext: true titleUrlDefault: https://www.tandfonline.com providerName: Taylor & Francis
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3NS8MwFA8yBL34LU6nRPAabZp2bY8iDg8yBCfuVpo20eLaydoJ_ve-l6bDIbKDHnpIm9eGvHy8l_7e7xFyAUZ3yB3NmfQzxTxwmFmSeC7rSyVSV0fclUbT98FwGI7H0YNFE1YWVok-tG6IIsxajZM7kVWLiLtCSjoOlgp4dy7GUoWRLzCEDyx7BPWNBs-LtRjM4SZgGok4QaSN4fntLUu70xJ36Y-12mxAg-1_aPoO2bLWJ71uhssuWVPlHtmwidBfP_fJ22ORTCZ0KvF4hmaqNkitkuYlnSnQqqIVIt7LF5oXsBJVFHfBjEIN5Ok0yElaKIwmzquCQjOpQSyyCoaColoZGlGq53hGd0CeBrejmztm8zGwVAS8Zq70whRqauQw6zvg2CWphyAP7XmBREdb6Qwu7qSOTKHoSbjvg8K1ypBE5pB0ymmpjgjlmXL8kHMZYB5yKZBlzwkzocB_kyISXeK1eohTS1aOOTMmMW85TW1PxtiTse3JLrlciL03bB2rBKLvSo5rc0yim5wmsVgh22tHRGwnfhW7fbC3wWXmQZecLx7DlMX_MEmppnOoA_aAG6Kze_yHz5-QTSwa8KXokU49m6tTsp5-1Hk1OzPT4AsLYwFP
linkProvider	Taylor & Francis
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT8MwDLZgIMGFN2I8g8S10DTd2h4RYgIxdmEIblHTJjCxdWjtkPj32Gk7gRDiAIce2iRtFOfhz7U_A5yg0h1y13BHtVLt-AiYnTj2PaettEg8E3FPWUl3g14vfHyMPsfCkFslYWhTEkXYvZoWNxmja5e4M-Kk46iqILzzKJgqjFoimocFyk5HAKzfeZjtxqgQlyHTRMWJbeoonp9e8-V8-sJe-m23tkdQZ_U_Or8GK5UCys7LGbMOczrbgKUqF_rz-ya83I3i4ZCNFVloWKoL66yVsUHGJhoFq1lOTu_ZExuMcDPKGR2EKcMaRNVpnSfZSFNA8SAfMewns06LTo6zQTOjLZMoM1My023Bfeeyf3HlVCkZnEQEvHA85YcJ1jREY9Z2EdvFiU9-Hsb3A0VYW5sUL-4mrkrw1lf4vIUyNzolHpltaGTjTO8A46l2WyHnKqBU5EoQ0Z4bpkIjhFMiEk3wa0HIpOIrp7QZQ8lrWtNqJCWNpKxGsgmns2avJWHHbw2iz1KWhbWUmDKtiRS_tN2vp4Ss1n4uvTaq3IiaedCE41kxrlr6FRNnejzFOqgSeCHh3d0_fP4Ilq76t13Zve7d7MEyFVlfTLEPjWIy1QewmLwVg3xyaNfEB0nFBXI
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3fa9swED66dmx9afejY2nTTYO9erMsO7YfS9vQshICy1jehGVLW2jslNgZ9L_vnSyHhVHy0D34wZZkC91JupO_-w7gMxrdCfcN91RUaC9Eh9nLsjDwBkqLPDApD5SV9E08GiXTaTp2aMLawSrJhzYtUYRdq2ly3xWmQ8R9JUo6jpYKencBxVIlaSTSZ7CHpnNEij0Z_lwvxmgPtxHTxMSJbbognsdes7E9bZCX_rNY2x1oePgf-v4KDpz5yc5afXkNO7p6Ay9dJvTf92_h9nuZzedsoeh8hhW6sVCtis0qttQoVs1qgrxXv9isxKWoZrQNFgxrEFGnhU6yUlM48awuGXaTWciiV6MuaGa05RFlZkWHdEfwY3g5Ob_yXEIGLxcxb7xAhUmONQ2RmA189OyyPCSUhwnDWJGnrU2BF_dzX-V4Gyp8HqHEjS6IReYd7FaLSr8HxgvtRwnnKqZE5EoQzZ6fFEKjA6dEKnoQdnKQuWMrp6QZc8k7UlM3kpJGUrqR7MGXdbO7lq5jW4P0byHLxp6TmDapiRRb2vY7jZBu5tcyGKDBjT4zj3vwaV2Mc5Z-xGSVXqywDhoEQULe7vETPv8RXowvhvLmevTtBPapxAIxRR92m-VKn8Lz_E8zq5cf7Ix4ACWFBCQ
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Small+object+detection+in+remote+sensing+images+based+on+attention+mechanism+and+multi-scale+feature+fusion&rft.jtitle=International+journal+of+remote+sensing&rft.au=Zhang%2C+Li-guo&rft.au=Wang%2C+Lei&rft.au=Jin%2C+Mei&rft.au=Geng%2C+Xing-shuo&rft.date=2022-05-03&rft.issn=1366-5901&rft.volume=43&rft.issue=9+p.3280-3297&rft.spage=3280&rft.epage=3297&rft_id=info:doi/10.1080%2F01431161.2022.2089539&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0143-1161&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0143-1161&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0143-1161&client=summon