Multimodal Representation Learning for Place Recognition Using Deep Hebbian Predictive Coding

Recognising familiar places is a competence required in many engineering applications that interact with the real world such as robot navigation. Combining information from different sensory sources promotes robustness and accuracy of place recognition. However, mismatch in data registration, dimens...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Frontiers in robotics and AI Ročník 8; s. 732023
Hlavní autori: Pearson, Martin J., Dora, Shirin, Struckmeier, Oliver, Knowles, Thomas C., Mitchinson, Ben, Tiwari, Kshitij, Kyrki, Ville, Bohte, Sander, Pennartz, Cyriel M.A.
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Switzerland Frontiers Media S.A 13.12.2021
Predmet:
multisensory integration
place recognition
predictive coding
Robotics and AI
sensory reconstruction
whisker tactile
multisensory integration
predictive coding
whisker tactile
place recognition
sensory reconstruction
ISSN:2296-9144, 2296-9144
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Recognising familiar places is a competence required in many engineering applications that interact with the real world such as robot navigation. Combining information from different sensory sources promotes robustness and accuracy of place recognition. However, mismatch in data registration, dimensionality, and timing between modalities remain challenging problems in multisensory place recognition. Spurious data generated by sensor drop-out in multisensory environments is particularly problematic and often resolved through adhoc and brittle solutions. An effective approach to these problems is demonstrated by animals as they gracefully move through the world. Therefore, we take a neuro-ethological approach by adopting self-supervised representation learning based on a neuroscientific model of visual cortex known as predictive coding. We demonstrate how this parsimonious network algorithm which is trained using a local learning rule can be extended to combine visual and tactile sensory cues from a biomimetic robot as it naturally explores a visually aliased environment. The place recognition performance obtained using joint latent representations generated by the network is significantly better than contemporary representation learning techniques. Further, we see evidence of improved robustness at place recognition in face of unimodal sensor drop-out. The proposed multimodal deep predictive coding algorithm presented is also linearly extensible to accommodate more than two sensory modalities, thereby providing an intriguing example of the value of neuro-biologically plausible representation learning for multimodal navigation.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Edited by: Teresa Vidal-Calleja, University of Technology Sydney, Australia
Reviewed by: Konstantinos Chatzilygeroudis, University of Patras, Greece
This article was submitted to Field Robotics, a section of the journal Frontiers in Robotics and AI
Riccardo Giubilato, Helmholtz Association of German Research Centers (HZ), Germany
ISSN:2296-9144
2296-9144
DOI:10.3389/frobt.2021.732023