Weld seam object detection system based on the fusion of 2D images and 3D point clouds using interpretable neural networks

This study introduces a novel approach that addresses the limitations of existing methods by integrating 2D image processing with 3D point cloud analysis, enhanced by interpretable neural networks. Unlike traditional methods that rely on either 2D or 3D data alone, our approach leverages the complem...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Scientific reports Ročník 14; číslo 1; s. 21137 - 13
Hlavní autori: Wang, Shengbo, Li, Zengxu, Chen, Guodong, Yue, Yaobin
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: London Nature Publishing Group UK 10.09.2024
Nature Publishing Group
Nature Portfolio
Predmet:
639/166/987
639/705/117
Decision making
Deep learning
Humanities and Social Sciences
Image processing
Industrial applications
Information processing
multidisciplinary
Neural networks
Pattern recognition
Science
Science (multidisciplinary)
Technicians
Welding
ISSN:2045-2322, 2045-2322
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This study introduces a novel approach that addresses the limitations of existing methods by integrating 2D image processing with 3D point cloud analysis, enhanced by interpretable neural networks. Unlike traditional methods that rely on either 2D or 3D data alone, our approach leverages the complementary strengths of both data types to improve detection accuracy in environments adversely affected by welding spatter and smoke. Our system employs an improved Faster R-CNN model with a ResNet50 backbone for 2D image analysis, coupled with an innovative orthogonal plane intersection line extraction algorithm for 3D point cloud processing. By incorporating explainable components such as visualizable feature maps and a transparent region proposal network, we address the “black box” issue common in deep learning models.This architecture enables a more transparent decision-making process, providing technicians with necessary insights to understand and trust the system’s outputs. The Faster-RCNN structure is designed to break down the object detection process into distinct, understandable steps, from initial feature extraction to final bounding box refinement. This fusion of 2D-3D data analysis and interpretability not only improves detection performance but also sets a new standard for transparency and reliability in automated welding systems, facilitating wider adoption in industrial applications.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-024-71989-w