DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic Segmentation
Semantic segmentation of aerial images is crucial yet resource-intensive. Inspired by human ability to learn rapidly, few-shot semantic segmentation offers a promising solution by utilizing limited labeled data for efficient model training and generalization. However, the intrinsic complexities of a...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10746596/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841533430763880448 |
|---|---|
| author | Xiyu Qi Yidan Zhang Lei Wang Yifan Wu Yi Xin Zhan Chen Yunping Ge |
| author_facet | Xiyu Qi Yidan Zhang Lei Wang Yifan Wu Yi Xin Zhan Chen Yunping Ge |
| author_sort | Xiyu Qi |
| collection | DOAJ |
| description | Semantic segmentation of aerial images is crucial yet resource-intensive. Inspired by human ability to learn rapidly, few-shot semantic segmentation offers a promising solution by utilizing limited labeled data for efficient model training and generalization. However, the intrinsic complexities of aerial images, compounded by scarce samples, often result in inadequate feature representation and semantic ambiguity, detracting from the model's performance. In this article, we propose to tackle these challenging problems via dual semantic metric learning and multisemantic features fusion and introduce a novel few-shot segmentation Network (DSMF-Net). On the one hand, we consider the inherent semantic gap between the feature of graph and grid structures and metric learning of few-shot segmentation. To exploit multiscale global semantic context, we construct scale-aware graph prototypes from different stages of the feature layers based on graph convolutional networks (GCNs), while also incorporating prior-guided metric learning to further enhance context at the high-level convolution features. On the other hand, we design a pyramid-based fusion and condensation mechanism to adaptively merge and couple the multisemantic information from support and query images. The indication and fusion of different semantic features can effectively emphasize the representation and coupling abilities of the network. We have conducted extensive experiments over the challenging iSAID-5<inline-formula><tex-math notation="LaTeX">$^{i}$</tex-math></inline-formula> and DLRSD benchmarks. The experiments have demonstrated our network's effectiveness and efficiency, yielding on-par performance with the state-of-the-art methods. |
| format | Article |
| id | doaj-art-aaaca3a28fe34d469546abc6e7f38aef |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-aaaca3a28fe34d469546abc6e7f38aef2025-01-16T00:00:31ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-011885386410.1109/JSTARS.2024.349360610746596DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic SegmentationXiyu Qi0https://orcid.org/0000-0001-6427-2812Yidan Zhang1https://orcid.org/0000-0002-7466-0234Lei Wang2Yifan Wu3https://orcid.org/0000-0003-2873-325XYi Xin4https://orcid.org/0009-0006-4668-6329Zhan Chen5Yunping Ge6https://orcid.org/0009-0003-2803-0869Key Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaKey Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaSemantic segmentation of aerial images is crucial yet resource-intensive. Inspired by human ability to learn rapidly, few-shot semantic segmentation offers a promising solution by utilizing limited labeled data for efficient model training and generalization. However, the intrinsic complexities of aerial images, compounded by scarce samples, often result in inadequate feature representation and semantic ambiguity, detracting from the model's performance. In this article, we propose to tackle these challenging problems via dual semantic metric learning and multisemantic features fusion and introduce a novel few-shot segmentation Network (DSMF-Net). On the one hand, we consider the inherent semantic gap between the feature of graph and grid structures and metric learning of few-shot segmentation. To exploit multiscale global semantic context, we construct scale-aware graph prototypes from different stages of the feature layers based on graph convolutional networks (GCNs), while also incorporating prior-guided metric learning to further enhance context at the high-level convolution features. On the other hand, we design a pyramid-based fusion and condensation mechanism to adaptively merge and couple the multisemantic information from support and query images. The indication and fusion of different semantic features can effectively emphasize the representation and coupling abilities of the network. We have conducted extensive experiments over the challenging iSAID-5<inline-formula><tex-math notation="LaTeX">$^{i}$</tex-math></inline-formula> and DLRSD benchmarks. The experiments have demonstrated our network's effectiveness and efficiency, yielding on-par performance with the state-of-the-art methods.https://ieeexplore.ieee.org/document/10746596/Aerial imagedual metric learningfew-shot learninggraph convolutional network (GCN)semantic segmentation |
| spellingShingle | Xiyu Qi Yidan Zhang Lei Wang Yifan Wu Yi Xin Zhan Chen Yunping Ge DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic Segmentation IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Aerial image dual metric learning few-shot learning graph convolutional network (GCN) semantic segmentation |
| title | DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic Segmentation |
| title_full | DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic Segmentation |
| title_fullStr | DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic Segmentation |
| title_full_unstemmed | DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic Segmentation |
| title_short | DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-Shot Aerial Image Semantic Segmentation |
| title_sort | dsmf net dual semantic metric learning fusion network for few shot aerial image semantic segmentation |
| topic | Aerial image dual metric learning few-shot learning graph convolutional network (GCN) semantic segmentation |
| url | https://ieeexplore.ieee.org/document/10746596/ |
| work_keys_str_mv | AT xiyuqi dsmfnetdualsemanticmetriclearningfusionnetworkforfewshotaerialimagesemanticsegmentation AT yidanzhang dsmfnetdualsemanticmetriclearningfusionnetworkforfewshotaerialimagesemanticsegmentation AT leiwang dsmfnetdualsemanticmetriclearningfusionnetworkforfewshotaerialimagesemanticsegmentation AT yifanwu dsmfnetdualsemanticmetriclearningfusionnetworkforfewshotaerialimagesemanticsegmentation AT yixin dsmfnetdualsemanticmetriclearningfusionnetworkforfewshotaerialimagesemanticsegmentation AT zhanchen dsmfnetdualsemanticmetriclearningfusionnetworkforfewshotaerialimagesemanticsegmentation AT yunpingge dsmfnetdualsemanticmetriclearningfusionnetworkforfewshotaerialimagesemanticsegmentation |