Dynamic Tuning and Multi-Task Learning-Based Model for Multimodal Sentiment Analysis
Multimodal sentiment analysis aims to uncover human affective states by integrating data from multiple sensory sources. However, previous studies have focused on optimizing model architecture, neglecting the impact of objective function settings on model performance. Given this, this study introduce...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/11/6342 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Multimodal sentiment analysis aims to uncover human affective states by integrating data from multiple sensory sources. However, previous studies have focused on optimizing model architecture, neglecting the impact of objective function settings on model performance. Given this, this study introduces a new framework, DMMSA, which utilizes the intrinsic correlation of sentiment signals and enhances the model’s understanding of complex sentiments. DMMSA incorporates coarse-grained sentiment analysis to reduce task complexity. Meanwhile, it embeds a contrastive learning mechanism within the modality, which decomposes unimodal features into similar and dissimilar ones, thus allowing for the simultaneous consideration of both unimodal and multimodal emotions. We tested DMMSA on the CH-SIMS, MOSI, and MOEI datasets. When only changing the optimization objectives, DMMSA achieved accuracy gains of 3.2%, 1.57%, and 1.95% over the baseline in five-class and seven-class classification tasks. In regression tasks, DMMSA reduced the Mean Absolute Error (MAE) by 1.46%, 1.5%, and 2.8% compared to the baseline. |
|---|---|
| ISSN: | 2076-3417 |