Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification
Ship image classification identifies the type of ships in an input image, which plays a significant role in the marine field. To enhance the ship classification performance, various research focuses on studying multi-modal ship classification, which aims at combining the advantages of visible images...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/22/7298 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846152459885477888 |
|---|---|
| author | Xin Song Zhikui Chen Fangming Zhong Jing Gao Jianning Zhang Peng Li |
| author_facet | Xin Song Zhikui Chen Fangming Zhong Jing Gao Jianning Zhang Peng Li |
| author_sort | Xin Song |
| collection | DOAJ |
| description | Ship image classification identifies the type of ships in an input image, which plays a significant role in the marine field. To enhance the ship classification performance, various research focuses on studying multi-modal ship classification, which aims at combining the advantages of visible images and infrared images to capture complementary information. However, the current methods simply concatenate features of different modalities to learn complementary information, which neglects the multi-level correlation between different modalities. Moreover, the existing methods require a large amount of labeled ship images to train the model. How to capture the multi-level cross-modal correlation between unlabeled and labeled data is still a challenge. In this paper, a novel semi-supervised multi-modal ship classification approach is proposed to solve these issues, which consists of two components, i.e., multi-level cross-modal interactive network and semi-supervised contrastive learning strategy. To learn comprehensive complementary information for classification, the multi-level cross-modal interactive network is designed to build local-level and global-level cross-modal feature correlation. Then, the semi-supervised contrastive learning strategy is employed to drive the optimization of the network with the intra-class consistency constraint based on supervision signals of unlabeled samples and prior label information. Extensive experiments on the public datasets demonstrate that our approach achieves state-of-the-art semi-supervised classification effectiveness. |
| format | Article |
| id | doaj-art-e806afdb2b8d4a91aa8bf4c6d33bb203 |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-e806afdb2b8d4a91aa8bf4c6d33bb2032024-11-26T18:21:26ZengMDPI AGSensors1424-82202024-11-012422729810.3390/s24227298Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship ClassificationXin Song0Zhikui Chen1Fangming Zhong2Jing Gao3Jianning Zhang4Peng Li5The School of Software Technology, Dalian University of Technology, Dalian 116621, ChinaThe School of Software Technology, Dalian University of Technology, Dalian 116621, ChinaThe School of Software Technology, Dalian University of Technology, Dalian 116621, ChinaThe School of Software Technology, Dalian University of Technology, Dalian 116621, ChinaThe School of Software Technology, Dalian University of Technology, Dalian 116621, ChinaThe School of Software Technology, Dalian University of Technology, Dalian 116621, ChinaShip image classification identifies the type of ships in an input image, which plays a significant role in the marine field. To enhance the ship classification performance, various research focuses on studying multi-modal ship classification, which aims at combining the advantages of visible images and infrared images to capture complementary information. However, the current methods simply concatenate features of different modalities to learn complementary information, which neglects the multi-level correlation between different modalities. Moreover, the existing methods require a large amount of labeled ship images to train the model. How to capture the multi-level cross-modal correlation between unlabeled and labeled data is still a challenge. In this paper, a novel semi-supervised multi-modal ship classification approach is proposed to solve these issues, which consists of two components, i.e., multi-level cross-modal interactive network and semi-supervised contrastive learning strategy. To learn comprehensive complementary information for classification, the multi-level cross-modal interactive network is designed to build local-level and global-level cross-modal feature correlation. Then, the semi-supervised contrastive learning strategy is employed to drive the optimization of the network with the intra-class consistency constraint based on supervision signals of unlabeled samples and prior label information. Extensive experiments on the public datasets demonstrate that our approach achieves state-of-the-art semi-supervised classification effectiveness.https://www.mdpi.com/1424-8220/24/22/7298ship classificationdeep multi-modal learningsemi-supervised learning |
| spellingShingle | Xin Song Zhikui Chen Fangming Zhong Jing Gao Jianning Zhang Peng Li Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification Sensors ship classification deep multi-modal learning semi-supervised learning |
| title | Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification |
| title_full | Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification |
| title_fullStr | Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification |
| title_full_unstemmed | Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification |
| title_short | Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification |
| title_sort | multi level cross modal interactive network based semi supervised multi modal ship classification |
| topic | ship classification deep multi-modal learning semi-supervised learning |
| url | https://www.mdpi.com/1424-8220/24/22/7298 |
| work_keys_str_mv | AT xinsong multilevelcrossmodalinteractivenetworkbasedsemisupervisedmultimodalshipclassification AT zhikuichen multilevelcrossmodalinteractivenetworkbasedsemisupervisedmultimodalshipclassification AT fangmingzhong multilevelcrossmodalinteractivenetworkbasedsemisupervisedmultimodalshipclassification AT jinggao multilevelcrossmodalinteractivenetworkbasedsemisupervisedmultimodalshipclassification AT jianningzhang multilevelcrossmodalinteractivenetworkbasedsemisupervisedmultimodalshipclassification AT pengli multilevelcrossmodalinteractivenetworkbasedsemisupervisedmultimodalshipclassification |