Multi-modal robotic visual-tactile localisation and detection of surface cracks

We present and validate a method to detect surface cracks with visual and tactile sensing. The proposed algorithm localises cracks in remote environments through videos/photos taken by an on-board robot camera. The identified areas of interest are then explored by a robot with a tactile sensor. Fast...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE International Conference on Automation Science and Engineering (CASE) S. 1806 - 1811
Hauptverfasser: Palermo, Francesca, Rincon-Ardila, Liz, Oh, Changjae, Althoefer, Kaspar, Poslad, Stefan, Venture, Gentiane, Farkhatdinov, Ildar
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 23.08.2021
Schlagworte:
Object detection
Optical fiber sensors
Optical fibers
Robot vision systems
Tactile sensors
Ultraviolet sources
Visualization
ISSN:2161-8089
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We present and validate a method to detect surface cracks with visual and tactile sensing. The proposed algorithm localises cracks in remote environments through videos/photos taken by an on-board robot camera. The identified areas of interest are then explored by a robot with a tactile sensor. Faster R-CNN object detection is used for identifying the location of potential cracks. Random forest classifier is used for tactile identification of the cracks to confirm their presences. Offline and online experiments to compare vision only and combined vision and tactile based crack detection are demonstrated. Two experiments are developed to test the efficiency of the multi-modal approach: online accuracy detection and time required to explore a surface and localise a crack. Exploring a cracked surface using combined visual and tactile modalities required four times less time than using the tactile modality only. The accuracy of detection was also improved with the combination of the two modalities. This approach may be implemented also in extreme environments since gamma radiation does not interfere with the sensing mechanism of fibre optic-based sensors.
ISSN:2161-8089
DOI:10.1109/CASE49439.2021.9551553