Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks
Satellite image generation has a wide range of applications. For example, parts of images must be restored in areas obscured by clouds or cloud shadows or areas that must be anonymized. The need to cover a large area with the generated images faces the challenge that separately generated images must...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | ISPRS International Journal of Geo-Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2220-9964/13/12/448 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846104334746517504 |
|---|---|
| author | Arminas Šidlauskas Andrius Kriščiūnas Dalia Čalnerytė |
| author_facet | Arminas Šidlauskas Andrius Kriščiūnas Dalia Čalnerytė |
| author_sort | Arminas Šidlauskas |
| collection | DOAJ |
| description | Satellite image generation has a wide range of applications. For example, parts of images must be restored in areas obscured by clouds or cloud shadows or areas that must be anonymized. The need to cover a large area with the generated images faces the challenge that separately generated images must maintain the structural and color continuity between the adjacent generated images as well as the actual ones. This study presents a modified architecture of the generative adversarial network (GAN) pix2pix that ensures the integrity of the generated remote sensing images. The pix2pix model comprises a U-Net generator and a PatchGAN discriminator. The generator was modified by expanding the input set with images representing the known parts of ground truth and the respective mask. Data used for the generative model consist of Sentinel-2 (S2) RGB satellite imagery as the target data and OpenStreetMap mapping data as the input. Since forested areas and fields dominate in images, a Kneedle clusterization method was applied to create datasets that better represent the other classes, such as buildings and roads. The original and updated models were trained on different datasets and their results were evaluated using gradient magnitude (GM), Fréchet inception distance (FID), structural similarity index measure (SSIM), and multiscale structural similarity index measure (MS-SSIM) metrics. The models with the updated architecture show improvement in gradient magnitude, SSIM, and MS-SSIM values for all datasets. The average GMs of the junction region and the full image are similar (do not exceed 7%) for the images generated using the modified architecture whereas it is more than 13% higher in the junction area for the images generated using the original architecture. The importance of class balancing is demonstrated by the fact that, for both architectures, models trained on the dataset with a higher ratio of classes representing buildings and roads compared to the models trained on the dataset without clusterization have more than 10% lower FID (162.673 to 190.036 for pix2pix and 173.408 to 195.621 for the modified architecture) and more than 5% higher SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) and MS-SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) values. |
| format | Article |
| id | doaj-art-0fa2e584844442589a8d50b1314b19f1 |
| institution | Kabale University |
| issn | 2220-9964 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | ISPRS International Journal of Geo-Information |
| spelling | doaj-art-0fa2e584844442589a8d50b1314b19f12024-12-27T14:30:08ZengMDPI AGISPRS International Journal of Geo-Information2220-99642024-12-01131244810.3390/ijgi13120448Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial NetworksArminas Šidlauskas0Andrius Kriščiūnas1Dalia Čalnerytė2Department of Applied Informatics, Faculty of Informatics, Kaunas University of Technology, Studentu St. 50, LT-51368 Kaunas, LithuaniaDepartment of Applied Informatics, Faculty of Informatics, Kaunas University of Technology, Studentu St. 50, LT-51368 Kaunas, LithuaniaDepartment of Applied Informatics, Faculty of Informatics, Kaunas University of Technology, Studentu St. 50, LT-51368 Kaunas, LithuaniaSatellite image generation has a wide range of applications. For example, parts of images must be restored in areas obscured by clouds or cloud shadows or areas that must be anonymized. The need to cover a large area with the generated images faces the challenge that separately generated images must maintain the structural and color continuity between the adjacent generated images as well as the actual ones. This study presents a modified architecture of the generative adversarial network (GAN) pix2pix that ensures the integrity of the generated remote sensing images. The pix2pix model comprises a U-Net generator and a PatchGAN discriminator. The generator was modified by expanding the input set with images representing the known parts of ground truth and the respective mask. Data used for the generative model consist of Sentinel-2 (S2) RGB satellite imagery as the target data and OpenStreetMap mapping data as the input. Since forested areas and fields dominate in images, a Kneedle clusterization method was applied to create datasets that better represent the other classes, such as buildings and roads. The original and updated models were trained on different datasets and their results were evaluated using gradient magnitude (GM), Fréchet inception distance (FID), structural similarity index measure (SSIM), and multiscale structural similarity index measure (MS-SSIM) metrics. The models with the updated architecture show improvement in gradient magnitude, SSIM, and MS-SSIM values for all datasets. The average GMs of the junction region and the full image are similar (do not exceed 7%) for the images generated using the modified architecture whereas it is more than 13% higher in the junction area for the images generated using the original architecture. The importance of class balancing is demonstrated by the fact that, for both architectures, models trained on the dataset with a higher ratio of classes representing buildings and roads compared to the models trained on the dataset without clusterization have more than 10% lower FID (162.673 to 190.036 for pix2pix and 173.408 to 195.621 for the modified architecture) and more than 5% higher SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) and MS-SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) values.https://www.mdpi.com/2220-9964/13/12/448remote sensingconditional generative adversarial networksatellite imageimage generation |
| spellingShingle | Arminas Šidlauskas Andrius Kriščiūnas Dalia Čalnerytė Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks ISPRS International Journal of Geo-Information remote sensing conditional generative adversarial network satellite image image generation |
| title | Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks |
| title_full | Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks |
| title_fullStr | Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks |
| title_full_unstemmed | Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks |
| title_short | Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks |
| title_sort | continuous satellite image generation from standard layer maps using conditional generative adversarial networks |
| topic | remote sensing conditional generative adversarial network satellite image image generation |
| url | https://www.mdpi.com/2220-9964/13/12/448 |
| work_keys_str_mv | AT arminassidlauskas continuoussatelliteimagegenerationfromstandardlayermapsusingconditionalgenerativeadversarialnetworks AT andriuskrisciunas continuoussatelliteimagegenerationfromstandardlayermapsusingconditionalgenerativeadversarialnetworks AT daliacalneryte continuoussatelliteimagegenerationfromstandardlayermapsusingconditionalgenerativeadversarialnetworks |