A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain Detection
A significant challenge that hinders advancements in medical research is the sensitive and confidential nature of patient data in available datasets. In particular, sharing patients’ facial images poses considerable privacy risks, especially with the rise of generative artificial intelligence (AI),...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Bioengineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2306-5354/12/2/195 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849719709020192768 |
|---|---|
| author | Mondher Bouazizi Kevin Feghoul Shengze Wang Yue Yin Tomoaki Ohtsuki |
| author_facet | Mondher Bouazizi Kevin Feghoul Shengze Wang Yue Yin Tomoaki Ohtsuki |
| author_sort | Mondher Bouazizi |
| collection | DOAJ |
| description | A significant challenge that hinders advancements in medical research is the sensitive and confidential nature of patient data in available datasets. In particular, sharing patients’ facial images poses considerable privacy risks, especially with the rise of generative artificial intelligence (AI), which could misuse such data if accessed by unauthorized parties. However, facial expressions are a valuable source of information for doctors and researchers, which creates a need for methods to derive them without compromising patient privacy or safety by exposing identifiable facial images. To address this, we present a quick, computationally efficient method for detecting action units (AUs) and their intensities—key indicators of health and emotion—using only 3D facial landmarks. Our proposed framework extracts 3D face landmarks from video recordings and employs a lightweight neural network (NN) to identify AUs and estimate AU intensities based on these landmarks. Our proposed method reaches a 79.25% F1-score in AU detection for the main AUs, and 0.66 in AU intensity estimation Root Mean Square Error (RMSE). This performance shows that it is possible for researchers to share 3D landmarks, which are far less intrusive, instead of facial images while maintaining high accuracy in AU detection. Moreover, to showcase the usefulness of our AU detection model, using the detected AUs and estimated intensities, we trained state-of-the-art Deep Learning (DL) models to detect pain. Our method reaches 91.16% accuracy in pain detection, which is not far behind the 93.14% accuracy obtained when employing a convolutional neural network (CNN) with residual blocks trained on actual images and the 92.11% accuracy obtained when employing all the ground-truth AUs. |
| format | Article |
| id | doaj-art-db63f96a8a704d9997c6ee73103b034c |
| institution | DOAJ |
| issn | 2306-5354 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Bioengineering |
| spelling | doaj-art-db63f96a8a704d9997c6ee73103b034c2025-08-20T03:12:05ZengMDPI AGBioengineering2306-53542025-02-0112219510.3390/bioengineering12020195A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain DetectionMondher Bouazizi0Kevin Feghoul1Shengze Wang2Yue Yin3Tomoaki Ohtsuki4Faculty of Science and Technology, Keio University, Yokohama 223-8522, JapanGraduate School of Science and Technology, Keio University, Yokohama 223-8522, JapanGraduate School of Science and Technology, Keio University, Yokohama 223-8522, JapanFaculty of Science and Technology, Keio University, Yokohama 223-8522, JapanFaculty of Science and Technology, Keio University, Yokohama 223-8522, JapanA significant challenge that hinders advancements in medical research is the sensitive and confidential nature of patient data in available datasets. In particular, sharing patients’ facial images poses considerable privacy risks, especially with the rise of generative artificial intelligence (AI), which could misuse such data if accessed by unauthorized parties. However, facial expressions are a valuable source of information for doctors and researchers, which creates a need for methods to derive them without compromising patient privacy or safety by exposing identifiable facial images. To address this, we present a quick, computationally efficient method for detecting action units (AUs) and their intensities—key indicators of health and emotion—using only 3D facial landmarks. Our proposed framework extracts 3D face landmarks from video recordings and employs a lightweight neural network (NN) to identify AUs and estimate AU intensities based on these landmarks. Our proposed method reaches a 79.25% F1-score in AU detection for the main AUs, and 0.66 in AU intensity estimation Root Mean Square Error (RMSE). This performance shows that it is possible for researchers to share 3D landmarks, which are far less intrusive, instead of facial images while maintaining high accuracy in AU detection. Moreover, to showcase the usefulness of our AU detection model, using the detected AUs and estimated intensities, we trained state-of-the-art Deep Learning (DL) models to detect pain. Our method reaches 91.16% accuracy in pain detection, which is not far behind the 93.14% accuracy obtained when employing a convolutional neural network (CNN) with residual blocks trained on actual images and the 92.11% accuracy obtained when employing all the ground-truth AUs.https://www.mdpi.com/2306-5354/12/2/1953D facial landmarksaction unitspain detectiontransformer |
| spellingShingle | Mondher Bouazizi Kevin Feghoul Shengze Wang Yue Yin Tomoaki Ohtsuki A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain Detection Bioengineering 3D facial landmarks action units pain detection transformer |
| title | A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain Detection |
| title_full | A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain Detection |
| title_fullStr | A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain Detection |
| title_full_unstemmed | A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain Detection |
| title_short | A Non-Invasive Approach for Facial Action Unit Extraction and Its Application in Pain Detection |
| title_sort | non invasive approach for facial action unit extraction and its application in pain detection |
| topic | 3D facial landmarks action units pain detection transformer |
| url | https://www.mdpi.com/2306-5354/12/2/195 |
| work_keys_str_mv | AT mondherbouazizi anoninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT kevinfeghoul anoninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT shengzewang anoninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT yueyin anoninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT tomoakiohtsuki anoninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT mondherbouazizi noninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT kevinfeghoul noninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT shengzewang noninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT yueyin noninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection AT tomoakiohtsuki noninvasiveapproachforfacialactionunitextractionanditsapplicationinpaindetection |