Multimodal geometric AutoEncoder (MGAE) for rail fasteners tightness evaluation with point clouds & monocular depth fusion - Qiu, Shi; Zaheer, Qasim; Muhammad Ahmed Hassan Shah, S.; Faizan Hussain Shah, Syed; Ehsan, Haleema; Atta, Zunaira; Ai, Chengbo; Wang, Jin; Wang, Weidong; Peng, Jun (Measurement : journal of the International Measurement Confederation, 2025)
•Introduces a Multimodal Geometric AutoEncoder (MGAE) for rail fastener tightness evaluation.•Integrates point cloud and monocular depth fusion for improved feature extraction.•Reduces manual annotation efforts and increases computational efficiency in rail inspection.•Provides a scalable, automated solution for rail infrastructure maintenance. Accurate detection and estimation of railway fastener tightness are vital for rail infrastructure safety and reliability. Traditional methods depend on manual annotation tools like Label Me, which are error-prone, labor-intensive, and costly. Additionally, monocular depth estimation and instance segmentation involve complex computations that challenge real-time implementation, particularly on resource-constrained platforms. This study introduces a novel three-phase solution using the Multimodal Geometric Autoencoder (MGAE) for fastener tightness detection, integrating point clouds with monocular-depth-guided multimodal data. Our approach utilizes a hybrid autoencoder for high-quality feature extraction, enabling precise tightness estimation. Employing unsupervised learning, MGAE eliminates the need for labeled data, thus reducing labor and costs. The framework integrates point clouds, mesh, monocular depth, and 2D images, with various fusion blocks enhancing feature extraction accuracy and computational efficiency. Post-feature extraction, classical techniques such as isolation forest, stress–strain, and elastic potential energy methods assess fastener tightness.