SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion Features
In comparison to conventional RGB cameras, the exceptional temporal resolution of event cameras allows them to capture rich information between frames, making them prime candidates for object tracking. Yet in practice, despite their theoretical advantages, the body of work on event-based multi-objec...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10816637/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841563403956518912 |
|---|---|
| author | Song Wang Zhu Wang Can Li Xiaojuan Qi Hayden Kwok-Hay So |
| author_facet | Song Wang Zhu Wang Can Li Xiaojuan Qi Hayden Kwok-Hay So |
| author_sort | Song Wang |
| collection | DOAJ |
| description | In comparison to conventional RGB cameras, the exceptional temporal resolution of event cameras allows them to capture rich information between frames, making them prime candidates for object tracking. Yet in practice, despite their theoretical advantages, the body of work on event-based multi-object tracking (MOT) remains in its infancy, especially in real-world environments where events from complex background and camera motion can easily obscure the true target motion. To address these limitations, we introduce SpikeMOT, an innovative event-based MOT framework employing spiking neural networks (SNNs) within a Siamese architecture. SpikeMOT extracts and associates sparse spatiotemporal features from event streams, enabling high-frequency object motion inference while preserving object identities. Additionally, a simultaneous object detector provides updated spatial information of these objects at an equivalent frame rate. To evaluate the efficacy of SpikeMOT, we present DSEC-MOT, a meticulously constructed, real-world event-based MOT benchmark. This dataset features manually corrected annotations for objects experiencing severe occlusions, frequent intersections, and out-of-view scenarios commonly encountered in real-world applications. Extensive experiments on the DSEC-MOT and the FE240hz dataset demonstrate SpikeMOT’s superior tracking accuracy under demanding conditions, advancing the state-of-the-art in event-based multi-object tracking. |
| format | Article |
| id | doaj-art-e8ccb3ef1f8740e4b10021f474f51a76 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e8ccb3ef1f8740e4b10021f474f51a762025-01-03T00:01:33ZengIEEEIEEE Access2169-35362025-01-011321423010.1109/ACCESS.2024.352341110816637SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion FeaturesSong Wang0https://orcid.org/0000-0002-1813-5865Zhu Wang1https://orcid.org/0000-0002-3859-1008Can Li2https://orcid.org/0000-0003-3795-2008Xiaojuan Qi3https://orcid.org/0000-0002-4285-1626Hayden Kwok-Hay So4https://orcid.org/0000-0002-6514-0237Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, ChinaDepartment of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, ChinaDepartment of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, ChinaDepartment of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, ChinaDepartment of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, ChinaIn comparison to conventional RGB cameras, the exceptional temporal resolution of event cameras allows them to capture rich information between frames, making them prime candidates for object tracking. Yet in practice, despite their theoretical advantages, the body of work on event-based multi-object tracking (MOT) remains in its infancy, especially in real-world environments where events from complex background and camera motion can easily obscure the true target motion. To address these limitations, we introduce SpikeMOT, an innovative event-based MOT framework employing spiking neural networks (SNNs) within a Siamese architecture. SpikeMOT extracts and associates sparse spatiotemporal features from event streams, enabling high-frequency object motion inference while preserving object identities. Additionally, a simultaneous object detector provides updated spatial information of these objects at an equivalent frame rate. To evaluate the efficacy of SpikeMOT, we present DSEC-MOT, a meticulously constructed, real-world event-based MOT benchmark. This dataset features manually corrected annotations for objects experiencing severe occlusions, frequent intersections, and out-of-view scenarios commonly encountered in real-world applications. Extensive experiments on the DSEC-MOT and the FE240hz dataset demonstrate SpikeMOT’s superior tracking accuracy under demanding conditions, advancing the state-of-the-art in event-based multi-object tracking.https://ieeexplore.ieee.org/document/10816637/Multi-object tracking (MOT)spiking neural networksevent-based MOT datasetsevent-based visionevent camera |
| spellingShingle | Song Wang Zhu Wang Can Li Xiaojuan Qi Hayden Kwok-Hay So SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion Features IEEE Access Multi-object tracking (MOT) spiking neural networks event-based MOT datasets event-based vision event camera |
| title | SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion Features |
| title_full | SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion Features |
| title_fullStr | SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion Features |
| title_full_unstemmed | SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion Features |
| title_short | SpikeMOT: Event-Based Multi-Object Tracking With Sparse Motion Features |
| title_sort | spikemot event based multi object tracking with sparse motion features |
| topic | Multi-object tracking (MOT) spiking neural networks event-based MOT datasets event-based vision event camera |
| url | https://ieeexplore.ieee.org/document/10816637/ |
| work_keys_str_mv | AT songwang spikemoteventbasedmultiobjecttrackingwithsparsemotionfeatures AT zhuwang spikemoteventbasedmultiobjecttrackingwithsparsemotionfeatures AT canli spikemoteventbasedmultiobjecttrackingwithsparsemotionfeatures AT xiaojuanqi spikemoteventbasedmultiobjecttrackingwithsparsemotionfeatures AT haydenkwokhayso spikemoteventbasedmultiobjecttrackingwithsparsemotionfeatures |