Ev-Conv: Fast CNN Inference on Event Camera Inputs for High-Speed Robot Perception

Event cameras capture visual information with a high temporal resolution and a wide dynamic range. This enables capturing visual information at fine time granularities (e.g., microseconds) in rapidly changing environments. This makes event cameras highly useful for high-speed robotics tasks involvin...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE robotics and automation letters Ročník 8; číslo 6; s. 3174 - 3181
Hlavní autori: Durvasula, Sankeerth, Guan, Yushi, Vijaykumar, Nandita
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Piscataway IEEE 01.06.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Artificial neural networks
Cameras
Changing environments
Computer architecture
computer architecture for robotic and automation
Convolution
Estimation
Floating point arithmetic
High speed
Inference
middleware and programming environments
Object recognition
Optical flow (image analysis)
Perception
Real time
Reduction
Robot vision systems
Robotics
software
Software architecture for robotic and automation
Sparsity
Task analysis
Temporal resolution
Tensors
Tracking control
ISSN:2377-3766, 2377-3766
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Event cameras capture visual information with a high temporal resolution and a wide dynamic range. This enables capturing visual information at fine time granularities (e.g., microseconds) in rapidly changing environments. This makes event cameras highly useful for high-speed robotics tasks involving rapid motion, such as high-speed perception, object tracking, and control. However, convolutional neural network inference on event camera streams cannot currently perform real-time inference at the high speeds at which event cameras operate-current CNN inference times are typically closer in order of magnitude to the frame rates of regular frame-based cameras. Real-time inference at event camera rates is necessary to fully leverage the high frequency and high temporal resolution that event cameras offer. This letter presents Ev-Conv, a new approach to enable fast inference on CNNs for inputs from event cameras. We observe that consecutive inputs to the CNN from an event camera have only small differences between them. Thus, we propose to perform inference on the difference between consecutive input tensors, or the increment . This enables a significant reduction in the number of floating-point operations required (and thus the inference latency) because increments are very sparse. We design Ev-Conv to leverage the irregular sparsity in increments from event cameras and to retain the sparsity of these increments across all layers of the network. We demonstrate a reduction in the number of floating operations required in the forward pass by up to 98%. We also demonstrate a speedup of up to <inline-formula><tex-math notation="LaTeX">1.6\times</tex-math></inline-formula> for inference using CNNs for tasks such as depth estimation, object recognition, and optical flow estimation, with almost no loss in accuracy.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2023.3259731