HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks
The objective of hyperspectral pansharpening is to fuse low-resolution hyperspectral images (LR-HSI) with corresponding panchromatic (PAN) images to generate high-resolution hyperspectral images (HR-HSI). Despite advancements in hyperspectral (HS) pansharpening using deep learning, the rich spectral...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/23/4389 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846123939746545664 |
|---|---|
| author | Jing Wang Xu Zhu Linhai Jing Yunwei Tang Hui Li Zhengqing Xiao Haifeng Ding |
| author_facet | Jing Wang Xu Zhu Linhai Jing Yunwei Tang Hui Li Zhengqing Xiao Haifeng Ding |
| author_sort | Jing Wang |
| collection | DOAJ |
| description | The objective of hyperspectral pansharpening is to fuse low-resolution hyperspectral images (LR-HSI) with corresponding panchromatic (PAN) images to generate high-resolution hyperspectral images (HR-HSI). Despite advancements in hyperspectral (HS) pansharpening using deep learning, the rich spectral details and large data volume of HS images place higher demands on models for effective spectral extraction and processing. In this paper, we present HyperGAN, a hyperspectral image fusion approach based on Generative Adversarial Networks. Unlike previous methods that deepen the network to capture spectral information, HyperGAN widens the structure with a Wide Block for multi-scale learning, effectively capturing global and local details from upsampled HSI and PAN images. While LR-HSI provides rich spectral data, PAN images offer spatial information. We introduce the Efficient Spatial and Channel Attention Module (ESCA) to integrate these features and add an energy-based discriminator to enhance model performance by learning directly from the Ground Truth (GT), improving fused image quality. We validated our method on various scenes, including the Pavia Center, Eastern Tianshan, and Chikusei. Results show that HyperGAN outperforms state-of-the-art methods in visual and quantitative evaluations. |
| format | Article |
| id | doaj-art-c43089af11aa4560bfbf900ab27a8856 |
| institution | Kabale University |
| issn | 2072-4292 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-c43089af11aa4560bfbf900ab27a88562024-12-13T16:30:40ZengMDPI AGRemote Sensing2072-42922024-11-011623438910.3390/rs16234389HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial NetworksJing Wang0Xu Zhu1Linhai Jing2Yunwei Tang3Hui Li4Zhengqing Xiao5Haifeng Ding6College of Mathematics and System Science, Xinjiang University, Urumqi 830017, ChinaCollege of Mathematics and System Science, Xinjiang University, Urumqi 830017, ChinaSchool of Information Engineering, China University of Geosciences (Beijing), Beijing 100083, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaCollege of Mathematics and System Science, Xinjiang University, Urumqi 830017, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaThe objective of hyperspectral pansharpening is to fuse low-resolution hyperspectral images (LR-HSI) with corresponding panchromatic (PAN) images to generate high-resolution hyperspectral images (HR-HSI). Despite advancements in hyperspectral (HS) pansharpening using deep learning, the rich spectral details and large data volume of HS images place higher demands on models for effective spectral extraction and processing. In this paper, we present HyperGAN, a hyperspectral image fusion approach based on Generative Adversarial Networks. Unlike previous methods that deepen the network to capture spectral information, HyperGAN widens the structure with a Wide Block for multi-scale learning, effectively capturing global and local details from upsampled HSI and PAN images. While LR-HSI provides rich spectral data, PAN images offer spatial information. We introduce the Efficient Spatial and Channel Attention Module (ESCA) to integrate these features and add an energy-based discriminator to enhance model performance by learning directly from the Ground Truth (GT), improving fused image quality. We validated our method on various scenes, including the Pavia Center, Eastern Tianshan, and Chikusei. Results show that HyperGAN outperforms state-of-the-art methods in visual and quantitative evaluations.https://www.mdpi.com/2072-4292/16/23/4389generative adversarial networkshyperspectral pansharpeningattentionenergy |
| spellingShingle | Jing Wang Xu Zhu Linhai Jing Yunwei Tang Hui Li Zhengqing Xiao Haifeng Ding HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks Remote Sensing generative adversarial networks hyperspectral pansharpening attention energy |
| title | HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks |
| title_full | HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks |
| title_fullStr | HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks |
| title_full_unstemmed | HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks |
| title_short | HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks |
| title_sort | hypergan a hyperspectral image fusion approach based on generative adversarial networks |
| topic | generative adversarial networks hyperspectral pansharpening attention energy |
| url | https://www.mdpi.com/2072-4292/16/23/4389 |
| work_keys_str_mv | AT jingwang hyperganahyperspectralimagefusionapproachbasedongenerativeadversarialnetworks AT xuzhu hyperganahyperspectralimagefusionapproachbasedongenerativeadversarialnetworks AT linhaijing hyperganahyperspectralimagefusionapproachbasedongenerativeadversarialnetworks AT yunweitang hyperganahyperspectralimagefusionapproachbasedongenerativeadversarialnetworks AT huili hyperganahyperspectralimagefusionapproachbasedongenerativeadversarialnetworks AT zhengqingxiao hyperganahyperspectralimagefusionapproachbasedongenerativeadversarialnetworks AT haifengding hyperganahyperspectralimagefusionapproachbasedongenerativeadversarialnetworks |