MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding
In terms of facial expressions, micro-expressions are more realistic than macro-expressions and provide more valuable information, which can be widely used in psychological counseling and clinical diagnosis. In the past few years, deep learning methods based on optical flow and Transformer have achi...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/23/7549 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849220446509072384 |
|---|---|
| author | Dehao Zhang Tao Zhang Haijiang Sun Yanhui Tang Qiaoyuan Liu |
| author_facet | Dehao Zhang Tao Zhang Haijiang Sun Yanhui Tang Qiaoyuan Liu |
| author_sort | Dehao Zhang |
| collection | DOAJ |
| description | In terms of facial expressions, micro-expressions are more realistic than macro-expressions and provide more valuable information, which can be widely used in psychological counseling and clinical diagnosis. In the past few years, deep learning methods based on optical flow and Transformer have achieved excellent results in this field, but most of the current algorithms are mainly concentrated on establishing a serialized token through the self-attention model, and they do not take into account the spatial relationship between facial landmarks. For the locality and changes in the micro-facial conditions themselves, we propose the deep learning model MCCA-VNET on the basis of Transformer. We effectively extract the changing features as the input of the model, fusing channel attention and spatial attention into Vision Transformer to capture correlations between features in different dimensions, which enhances the accuracy of the identification of micro-expressions. In order to verify the effectiveness of the algorithm mentioned, we conduct experimental testing in the SAMM, CAS (ME) II, and SMIC datasets and compared the results with other former best algorithms. Our algorithms can improve the UF1 score and UAR score to, respectively, 0.8676 and 0.8622 for the composite dataset, and they are better than other algorithms on multiple indicators, achieving the best comprehensive performance. |
| format | Article |
| id | doaj-art-2f89dbeacb5c483bb05f7c6e64f809b8 |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-2f89dbeacb5c483bb05f7c6e64f809b82024-12-13T16:31:54ZengMDPI AGSensors1424-82202024-11-012423754910.3390/s24237549MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial CodingDehao Zhang0Tao Zhang1Haijiang Sun2Yanhui Tang3Qiaoyuan Liu4Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaIn terms of facial expressions, micro-expressions are more realistic than macro-expressions and provide more valuable information, which can be widely used in psychological counseling and clinical diagnosis. In the past few years, deep learning methods based on optical flow and Transformer have achieved excellent results in this field, but most of the current algorithms are mainly concentrated on establishing a serialized token through the self-attention model, and they do not take into account the spatial relationship between facial landmarks. For the locality and changes in the micro-facial conditions themselves, we propose the deep learning model MCCA-VNET on the basis of Transformer. We effectively extract the changing features as the input of the model, fusing channel attention and spatial attention into Vision Transformer to capture correlations between features in different dimensions, which enhances the accuracy of the identification of micro-expressions. In order to verify the effectiveness of the algorithm mentioned, we conduct experimental testing in the SAMM, CAS (ME) II, and SMIC datasets and compared the results with other former best algorithms. Our algorithms can improve the UF1 score and UAR score to, respectively, 0.8676 and 0.8622 for the composite dataset, and they are better than other algorithms on multiple indicators, achieving the best comprehensive performance.https://www.mdpi.com/1424-8220/24/23/7549micro-expressionoptical flow methodfacial codingMCCA-VNETvision transformer |
| spellingShingle | Dehao Zhang Tao Zhang Haijiang Sun Yanhui Tang Qiaoyuan Liu MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding Sensors micro-expression optical flow method facial coding MCCA-VNET vision transformer |
| title | MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding |
| title_full | MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding |
| title_fullStr | MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding |
| title_full_unstemmed | MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding |
| title_short | MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding |
| title_sort | mcca vnet a vit based deep learning approach for micro expression recognition based on facial coding |
| topic | micro-expression optical flow method facial coding MCCA-VNET vision transformer |
| url | https://www.mdpi.com/1424-8220/24/23/7549 |
| work_keys_str_mv | AT dehaozhang mccavnetavitbaseddeeplearningapproachformicroexpressionrecognitionbasedonfacialcoding AT taozhang mccavnetavitbaseddeeplearningapproachformicroexpressionrecognitionbasedonfacialcoding AT haijiangsun mccavnetavitbaseddeeplearningapproachformicroexpressionrecognitionbasedonfacialcoding AT yanhuitang mccavnetavitbaseddeeplearningapproachformicroexpressionrecognitionbasedonfacialcoding AT qiaoyuanliu mccavnetavitbaseddeeplearningapproachformicroexpressionrecognitionbasedonfacialcoding |