Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution
Remote sensing image super-resolution (SR) is a crucial task to restore high-resolution (HR) images from low-resolution (LR) observations. Recently, the denoising diffusion probabilistic model (DDPM) has shown promising performance in image reconstructions by overcoming problems inherent in generati...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10763472/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846129687775936512 |
|---|---|
| author | Jialu Sui Xianping Ma Xiaokang Zhang Man-On Pun Hao Wu |
| author_facet | Jialu Sui Xianping Ma Xiaokang Zhang Man-On Pun Hao Wu |
| author_sort | Jialu Sui |
| collection | DOAJ |
| description | Remote sensing image super-resolution (SR) is a crucial task to restore high-resolution (HR) images from low-resolution (LR) observations. Recently, the denoising diffusion probabilistic model (DDPM) has shown promising performance in image reconstructions by overcoming problems inherent in generative models, such as oversmoothing and mode collapse. However, the high-frequency details generated by DDPM often suffer from misalignment with HR images due to the model's tendency to overlook long-range semantic contexts. This challenge is partly due to the prevalent use of a U-Net decoder in the conditional noise predictor, which favors local details and can introduce noise with considerable variance during prediction. To tackle these limitations, an adaptive semantic-enhanced DDPM (ASDDPM) is proposed to enhance the detail-preserving capability of the DDPM by integrating low-frequency semantic insights through a transformer. Specifically, a novel adaptive diffusion transformer decoder is developed to bridge the semantic gap between the encoder and decoder by regulating the noise prediction with the global contextual relationships and long-range dependencies in the diffusion process. In addition, a residual feature fusion strategy establishes information exchange between the two decoders at multiple levels. As a result, the predicted noise generated by our approach closely approximates that of the real noise distribution. Extensive experiments on two SR and two semantic segmentation datasets confirm the superior performance of the proposed ASDDPM in both SR and the subsequent downstream applications. |
| format | Article |
| id | doaj-art-6e919dba6a8e4c5bbc05d1e208a32713 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-6e919dba6a8e4c5bbc05d1e208a327132024-12-10T00:00:30ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-011889290610.1109/JSTARS.2024.350456910763472Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-ResolutionJialu Sui0https://orcid.org/0000-0002-7450-766XXianping Ma1https://orcid.org/0000-0002-2180-2964Xiaokang Zhang2https://orcid.org/0000-0002-6127-4801Man-On Pun3https://orcid.org/0000-0003-3316-5381Hao Wu4https://orcid.org/0000-0001-5751-7885School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, ChinaSchool of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, ChinaSchool of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan, ChinaSchool of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, ChinaCollege of Urban and Environment Sciences, Central China Normal University, Wuhan, ChinaRemote sensing image super-resolution (SR) is a crucial task to restore high-resolution (HR) images from low-resolution (LR) observations. Recently, the denoising diffusion probabilistic model (DDPM) has shown promising performance in image reconstructions by overcoming problems inherent in generative models, such as oversmoothing and mode collapse. However, the high-frequency details generated by DDPM often suffer from misalignment with HR images due to the model's tendency to overlook long-range semantic contexts. This challenge is partly due to the prevalent use of a U-Net decoder in the conditional noise predictor, which favors local details and can introduce noise with considerable variance during prediction. To tackle these limitations, an adaptive semantic-enhanced DDPM (ASDDPM) is proposed to enhance the detail-preserving capability of the DDPM by integrating low-frequency semantic insights through a transformer. Specifically, a novel adaptive diffusion transformer decoder is developed to bridge the semantic gap between the encoder and decoder by regulating the noise prediction with the global contextual relationships and long-range dependencies in the diffusion process. In addition, a residual feature fusion strategy establishes information exchange between the two decoders at multiple levels. As a result, the predicted noise generated by our approach closely approximates that of the real noise distribution. Extensive experiments on two SR and two semantic segmentation datasets confirm the superior performance of the proposed ASDDPM in both SR and the subsequent downstream applications.https://ieeexplore.ieee.org/document/10763472/Denoising diffusion probabilistic modelremote sensing imagessingle image super-resolution |
| spellingShingle | Jialu Sui Xianping Ma Xiaokang Zhang Man-On Pun Hao Wu Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Denoising diffusion probabilistic model remote sensing images single image super-resolution |
| title | Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution |
| title_full | Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution |
| title_fullStr | Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution |
| title_full_unstemmed | Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution |
| title_short | Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution |
| title_sort | adaptive semantic enhanced denoising diffusion probabilistic model for remote sensing image super resolution |
| topic | Denoising diffusion probabilistic model remote sensing images single image super-resolution |
| url | https://ieeexplore.ieee.org/document/10763472/ |
| work_keys_str_mv | AT jialusui adaptivesemanticenhanceddenoisingdiffusionprobabilisticmodelforremotesensingimagesuperresolution AT xianpingma adaptivesemanticenhanceddenoisingdiffusionprobabilisticmodelforremotesensingimagesuperresolution AT xiaokangzhang adaptivesemanticenhanceddenoisingdiffusionprobabilisticmodelforremotesensingimagesuperresolution AT manonpun adaptivesemanticenhanceddenoisingdiffusionprobabilisticmodelforremotesensingimagesuperresolution AT haowu adaptivesemanticenhanceddenoisingdiffusionprobabilisticmodelforremotesensingimagesuperresolution |