FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking
RGBT tracking technology often struggles in noncooperative challenges, such as illumination variation, scale variation, fast motion, occlusion, and thermal crossover. In this work, we propose a novel RGBT tracking model, called challenge-based Feature complementary Fusion Network (FcFNet), especiall...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10803956/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841563355731460096 |
|---|---|
| author | Wensheng Wang Congjian Li Di Zhang Huihui Zhou Mingli Xie Haoran Zhou Kun Fu |
| author_facet | Wensheng Wang Congjian Li Di Zhang Huihui Zhou Mingli Xie Haoran Zhou Kun Fu |
| author_sort | Wensheng Wang |
| collection | DOAJ |
| description | RGBT tracking technology often struggles in noncooperative challenges, such as illumination variation, scale variation, fast motion, occlusion, and thermal crossover. In this work, we propose a novel RGBT tracking model, called challenge-based Feature complementary Fusion Network (FcFNet), especially aiming at noncooperative scenarios. In particular, we first decouple the mixed challenging attributes and adaptively combine features at each layer using the challenge branch fusion module (CBFM), forming more discriminative target representations. Subsequently, we employ two nonshared convolution kernels in each layer and one shared convolution kernel to, respectively, extract individual and common features of different modalities. By applying the dynamic convolution fusion module (DCFM), the nonshared and shared convolution kernels are reweighted dynamically. Finally, we embed the feature-enhanced aggregation and dynamic kernel weights into the backbone network, which forms the FcFNet. Experimental evaluations conducted on the GTOT and RGBT234 datasets demonstrate a notable enhancement in detection performance, with an absolute improvement of 2.4% in success rate and 3.3% in precision rate when leveraging the CBFM and DCFM. The proposed FcFNet exhibits competitive performance compared with other state-of-the-art tracking algorithms, especially in continuous tracking tasks for noncooperative targets. |
| format | Article |
| id | doaj-art-3ec7b4f21023414ba066f4f15d627a61 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-3ec7b4f21023414ba066f4f15d627a612025-01-03T00:00:28ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01182239225110.1109/JSTARS.2024.351846010803956FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT TrackingWensheng Wang0https://orcid.org/0000-0002-4031-2303Congjian Li1Di Zhang2https://orcid.org/0009-0006-6940-4427Huihui Zhou3Mingli Xie4Haoran Zhou5Kun Fu6Key Laboratory of Network Information System Technology, Institute of Electronics, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Network Information System Technology, Institute of Electronics, Chinese Academy of Sciences, Beijing, ChinaNational University of Defense Technology, Changsha, ChinaNational Geomatics Center of China, Beijing, ChinaKey Laboratory of Network Information System Technology, Institute of Electronics, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Network Information System Technology, Institute of Electronics, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Network Information System Technology, Institute of Electronics, Chinese Academy of Sciences, Beijing, ChinaRGBT tracking technology often struggles in noncooperative challenges, such as illumination variation, scale variation, fast motion, occlusion, and thermal crossover. In this work, we propose a novel RGBT tracking model, called challenge-based Feature complementary Fusion Network (FcFNet), especially aiming at noncooperative scenarios. In particular, we first decouple the mixed challenging attributes and adaptively combine features at each layer using the challenge branch fusion module (CBFM), forming more discriminative target representations. Subsequently, we employ two nonshared convolution kernels in each layer and one shared convolution kernel to, respectively, extract individual and common features of different modalities. By applying the dynamic convolution fusion module (DCFM), the nonshared and shared convolution kernels are reweighted dynamically. Finally, we embed the feature-enhanced aggregation and dynamic kernel weights into the backbone network, which forms the FcFNet. Experimental evaluations conducted on the GTOT and RGBT234 datasets demonstrate a notable enhancement in detection performance, with an absolute improvement of 2.4% in success rate and 3.3% in precision rate when leveraging the CBFM and DCFM. The proposed FcFNet exhibits competitive performance compared with other state-of-the-art tracking algorithms, especially in continuous tracking tasks for noncooperative targets.https://ieeexplore.ieee.org/document/10803956/Challenge modelingcross-modal interactiondeep learningnoncooperative targetsRGBT tracking |
| spellingShingle | Wensheng Wang Congjian Li Di Zhang Huihui Zhou Mingli Xie Haoran Zhou Kun Fu FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Challenge modeling cross-modal interaction deep learning noncooperative targets RGBT tracking |
| title | FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking |
| title_full | FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking |
| title_fullStr | FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking |
| title_full_unstemmed | FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking |
| title_short | FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking |
| title_sort | fcfnet a challenge based feature complementary fusion network for rgbt tracking |
| topic | Challenge modeling cross-modal interaction deep learning noncooperative targets RGBT tracking |
| url | https://ieeexplore.ieee.org/document/10803956/ |
| work_keys_str_mv | AT wenshengwang fcfnetachallengebasedfeaturecomplementaryfusionnetworkforrgbttracking AT congjianli fcfnetachallengebasedfeaturecomplementaryfusionnetworkforrgbttracking AT dizhang fcfnetachallengebasedfeaturecomplementaryfusionnetworkforrgbttracking AT huihuizhou fcfnetachallengebasedfeaturecomplementaryfusionnetworkforrgbttracking AT minglixie fcfnetachallengebasedfeaturecomplementaryfusionnetworkforrgbttracking AT haoranzhou fcfnetachallengebasedfeaturecomplementaryfusionnetworkforrgbttracking AT kunfu fcfnetachallengebasedfeaturecomplementaryfusionnetworkforrgbttracking |