Spike-event Object Detection for Neuromorphic Vision

Neuromorphic vision is one of the novel research fields that studies neuromorphic cameras and spiking neural networks (SNNs) for computer vision. Instead of computing on frame-based images, spike events are streamed from neuromorphic cameras, and novel object detection algorithms have to deal with spike events to achieve detection tasks. In this paper, we propose a solution of the novel object detection method with spike events. Spike events are first decoded to event images according to the computational methodology of neuromorphic theory. The event images can be realized as change-detected images of moving objects with a high frame rate. A redesigned deep learning framework is proposed for the object detection to deal with the event images. We propose a deep SNN method that can be realized by the conversion of successful convolution neural networks but trained by event images. We also design a methodology to build event-image datasets by object tracking algorithms. The proposed solution therefore includes spike event decoding, a redesigned deep SNN, and an event-image dataset algorithm. Experiments are conducted not only on the MNIST-DVS dataset, which is a benchmark dataset for the study of neuromorphic vision, but also on our event pedestrian detection dataset. The experimental results show that the performance of our automatic labeling algorithm is close to the model trained on manual labeled data. Moreover, the algorithm efficiency can be further improved with the PAFBenchmark dataset. The model comparison result shows that our proposed model has higher accuracy than existing SNN methods, better energy efficiency, and lower energy consumption than existing CNN methods. It demonstrates that our deep SNN method is a feasible solution for the study of neuromorphic vision. The intuition that deep SNN trained with more learning data can achieve better accuracy is also confirmed in this brand new research field.