Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation Network
Image-to-image translation methods have advanced from focusing on image-level info to incorporating pixel-level and instance-level details. However, with feature-level constraint, deviation occurs when the network overemphasizes convolutional features, neglecting traditional image feature extraction...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/13/1/177 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549138389368832 |
|---|---|
| author | Hangyao Tu Zheng Wang Yanwei Zhao |
| author_facet | Hangyao Tu Zheng Wang Yanwei Zhao |
| author_sort | Hangyao Tu |
| collection | DOAJ |
| description | Image-to-image translation methods have advanced from focusing on image-level info to incorporating pixel-level and instance-level details. However, with feature-level constraint, deviation occurs when the network overemphasizes convolutional features, neglecting traditional image feature extraction. To address this, we proposed the multimodal image translation algorithm MASSE based on a Singular Squeeze-and-Excitation Network, combining GANs and SENet. It utilizes SVD features to assist the SENet in managing the scaling degree. The SENet employs SVD to extract features and enhance the Excitation operation to obtain new channel attention weights and form attention feature maps. Then, image content features are refined by combining convolutional and attention feature maps, and style features are obtained by the style generator. Finally, content and style features are combined to generate new style images. Ablation experiments showed the optimal SVD parameter is 128, producing the best translation results. According to FID, MASSE outperforms current methods in generating diverse images. |
| format | Article |
| id | doaj-art-50b8d4e9635d4c7abcef52d85e31f2f1 |
| institution | Kabale University |
| issn | 2227-7390 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj-art-50b8d4e9635d4c7abcef52d85e31f2f12025-01-10T13:18:32ZengMDPI AGMathematics2227-73902025-01-0113117710.3390/math13010177Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation NetworkHangyao Tu0Zheng Wang1Yanwei Zhao2School of Computer Science and Technology, Zhejiang University, Hangzhou 310015, ChinaSchool of Computer and Computational Science, Hangzhou City University, Hangzhou 310015, ChinaCollege of Engineering, Zhejiang University of Technology, Hangzhou 310015, ChinaImage-to-image translation methods have advanced from focusing on image-level info to incorporating pixel-level and instance-level details. However, with feature-level constraint, deviation occurs when the network overemphasizes convolutional features, neglecting traditional image feature extraction. To address this, we proposed the multimodal image translation algorithm MASSE based on a Singular Squeeze-and-Excitation Network, combining GANs and SENet. It utilizes SVD features to assist the SENet in managing the scaling degree. The SENet employs SVD to extract features and enhance the Excitation operation to obtain new channel attention weights and form attention feature maps. Then, image content features are refined by combining convolutional and attention feature maps, and style features are obtained by the style generator. Finally, content and style features are combined to generate new style images. Ablation experiments showed the optimal SVD parameter is 128, producing the best translation results. According to FID, MASSE outperforms current methods in generating diverse images.https://www.mdpi.com/2227-7390/13/1/177image translationgenerative modelsingular value decompositionmultimodal images |
| spellingShingle | Hangyao Tu Zheng Wang Yanwei Zhao Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation Network Mathematics image translation generative model singular value decomposition multimodal images |
| title | Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation Network |
| title_full | Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation Network |
| title_fullStr | Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation Network |
| title_full_unstemmed | Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation Network |
| title_short | Multimodal Image Translation Algorithm Based on Singular Squeeze-and-Excitation Network |
| title_sort | multimodal image translation algorithm based on singular squeeze and excitation network |
| topic | image translation generative model singular value decomposition multimodal images |
| url | https://www.mdpi.com/2227-7390/13/1/177 |
| work_keys_str_mv | AT hangyaotu multimodalimagetranslationalgorithmbasedonsingularsqueezeandexcitationnetwork AT zhengwang multimodalimagetranslationalgorithmbasedonsingularsqueezeandexcitationnetwork AT yanweizhao multimodalimagetranslationalgorithmbasedonsingularsqueezeandexcitationnetwork |