VRFfall: Cross Vision-RF Fall Detection with Camera and mmWave Radar

Accurate fall detection systems are vital to address the global health concern of elderly falls, which often lead to severe injuries, hospitalizations, and fatalities. Since falls can happen at any time in any location, it is imperative to have a comprehensive system that boasts high applicability a...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings - International Conference on Parallel and Distributed Systems pp. 100 - 107
Main Authors: Zhu, Yanying, Song, Haotian, Wu, Kaishun, Sun, Min, Zhou, Li
Format: Conference Proceeding
Language:English
Published: IEEE 10.10.2024
Subjects:
Cameras
Fall detection
Fuses
Human Sensing
Knowledge transfer
Millimeter wave communication
Multi-modal
Quality assessment
Radar
Radar detection
Sensor systems
Sensors
Wireless Sensing
ISSN:2690-5965
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Accurate fall detection systems are vital to address the global health concern of elderly falls, which often lead to severe injuries, hospitalizations, and fatalities. Since falls can happen at any time in any location, it is imperative to have a comprehensive system that boasts high applicability across a broad range of scenarios, operating seamlessly 24/7. However, within a range of fall detection systems, most of the existing work is built upon mono-modal sensors, which are inevitably inherited and constrained by mono-modal shortages. To overcome the constraints of mono-modal systems, we introduce VRFfall, a novel multi-modal fall detection system that seamlessly fuses mmWave radar and camera technologies. As a system with high generalization capabilities, VRFfall supports both multi-modal and mono-modal inputs with its independent feature extraction pipeline for each modality. Utilizing a cross-modal knowledge transfer design, VRFfall enhances performance with mono-modal input by leveraging fused knowledge from the other modality. Moreover, to ensure optimal fusion decisions under modal discrepancies, VRFfall incorporates an adaptive Modal Quality Assessment Module (MQAM) that dynamically evaluates and fuses features from both modalities. Extensive evaluations using a dataset collected from 20 volunteers across two environments and three conditions have been conducted on VRFfall. The results demonstrate its high performance and excellent generalization across diverse environments and conditions, promising a 24/7 continuous fall detection system.
ISSN:2690-5965
DOI:10.1109/ICPADS63350.2024.00023