Automated video-based pain recognition in cats using facial landmarks
Abstract Affective states are reflected in the facial expressions of all mammals. Facial behaviors linked to pain have attracted most of the attention so far in non-human animals, leading to the development of numerous instruments for evaluating pain through facial expressions for various animal spe...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-11-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-78406-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846165173956509696 |
|---|---|
| author | George Martvel Teddy Lazebnik Marcelo Feighelstein Lea Henze Sebastian Meller Ilan Shimshoni Friederike Twele Alexandra Schütter Nora Foraita Sabine Kästner Lauren Finka Stelio P. L. Luna Daniel S. Mills Holger A. Volk Anna Zamansky |
| author_facet | George Martvel Teddy Lazebnik Marcelo Feighelstein Lea Henze Sebastian Meller Ilan Shimshoni Friederike Twele Alexandra Schütter Nora Foraita Sabine Kästner Lauren Finka Stelio P. L. Luna Daniel S. Mills Holger A. Volk Anna Zamansky |
| author_sort | George Martvel |
| collection | DOAJ |
| description | Abstract Affective states are reflected in the facial expressions of all mammals. Facial behaviors linked to pain have attracted most of the attention so far in non-human animals, leading to the development of numerous instruments for evaluating pain through facial expressions for various animal species. Nevertheless, manual facial expression analysis is susceptible to subjectivity and bias, is labor-intensive and often necessitates specialized expertise and training. This challenge has spurred a growing body of research into automated pain recognition, which has been explored for multiple species, including cats. In our previous studies, we have presented and studied artificial intelligence (AI) pipelines for automated pain recognition in cats using 48 facial landmarks grounded in cats’ facial musculature, as well as an automated detector of these landmarks. However, so far automated recognition of pain in cats used solely static information obtained from hand-picked single images of good quality. This study takes a significant step forward in fully automated pain detection applications by presenting an end-to-end AI pipeline that requires no manual efforts in the selection of suitable images or their landmark annotation. By working with video rather than still images, this new pipeline approach also optimises the temporal dimension of visual information capture in a way that is not practical to preform manually. The presented pipeline reaches over 70% and 66% accuracy respectively in two different cat pain datasets, outperforming previous automated landmark-based approaches using single frames under similar conditions, indicating that dynamics matter in cat pain recognition. We further define metrics for measuring different dimensions of deficiencies in datasets with animal pain faces, and investigate their impact on the performance of the presented pain recognition AI pipeline. |
| format | Article |
| id | doaj-art-28a19f6ee2714e309df9cbb1078118b7 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-28a19f6ee2714e309df9cbb1078118b72024-11-17T12:30:07ZengNature PortfolioScientific Reports2045-23222024-11-0114111110.1038/s41598-024-78406-2Automated video-based pain recognition in cats using facial landmarksGeorge Martvel0Teddy Lazebnik1Marcelo Feighelstein2Lea Henze3Sebastian Meller4Ilan Shimshoni5Friederike Twele6Alexandra Schütter7Nora Foraita8Sabine Kästner9Lauren Finka10Stelio P. L. Luna11Daniel S. Mills12Holger A. Volk13Anna Zamansky14Information Systems Department, University of HaifaDepartment of Mathematics, Ariel UniversityInformation Systems Department, University of HaifaDepartment of Small Animal Medicine and Surgery, University of Veterinary Medicine HannoverDepartment of Small Animal Medicine and Surgery, University of Veterinary Medicine HannoverInformation Systems Department, University of HaifaDepartment of Small Animal Medicine and Surgery, University of Veterinary Medicine HannoverDepartment of Small Animal Medicine and Surgery, University of Veterinary Medicine HannoverDepartment of Small Animal Medicine and Surgery, University of Veterinary Medicine HannoverDepartment of Small Animal Medicine and Surgery, University of Veterinary Medicine HannoverCats Protection, National Cat Centre, Chelwood GateSchool of Veterinary Medicine and Animal Science, São Paulo State University (Unesp)School of Life & Environmental Sciences, Joseph Bank Laboratories, University of LincolnDepartment of Small Animal Medicine and Surgery, University of Veterinary Medicine HannoverInformation Systems Department, University of HaifaAbstract Affective states are reflected in the facial expressions of all mammals. Facial behaviors linked to pain have attracted most of the attention so far in non-human animals, leading to the development of numerous instruments for evaluating pain through facial expressions for various animal species. Nevertheless, manual facial expression analysis is susceptible to subjectivity and bias, is labor-intensive and often necessitates specialized expertise and training. This challenge has spurred a growing body of research into automated pain recognition, which has been explored for multiple species, including cats. In our previous studies, we have presented and studied artificial intelligence (AI) pipelines for automated pain recognition in cats using 48 facial landmarks grounded in cats’ facial musculature, as well as an automated detector of these landmarks. However, so far automated recognition of pain in cats used solely static information obtained from hand-picked single images of good quality. This study takes a significant step forward in fully automated pain detection applications by presenting an end-to-end AI pipeline that requires no manual efforts in the selection of suitable images or their landmark annotation. By working with video rather than still images, this new pipeline approach also optimises the temporal dimension of visual information capture in a way that is not practical to preform manually. The presented pipeline reaches over 70% and 66% accuracy respectively in two different cat pain datasets, outperforming previous automated landmark-based approaches using single frames under similar conditions, indicating that dynamics matter in cat pain recognition. We further define metrics for measuring different dimensions of deficiencies in datasets with animal pain faces, and investigate their impact on the performance of the presented pain recognition AI pipeline.https://doi.org/10.1038/s41598-024-78406-2 |
| spellingShingle | George Martvel Teddy Lazebnik Marcelo Feighelstein Lea Henze Sebastian Meller Ilan Shimshoni Friederike Twele Alexandra Schütter Nora Foraita Sabine Kästner Lauren Finka Stelio P. L. Luna Daniel S. Mills Holger A. Volk Anna Zamansky Automated video-based pain recognition in cats using facial landmarks Scientific Reports |
| title | Automated video-based pain recognition in cats using facial landmarks |
| title_full | Automated video-based pain recognition in cats using facial landmarks |
| title_fullStr | Automated video-based pain recognition in cats using facial landmarks |
| title_full_unstemmed | Automated video-based pain recognition in cats using facial landmarks |
| title_short | Automated video-based pain recognition in cats using facial landmarks |
| title_sort | automated video based pain recognition in cats using facial landmarks |
| url | https://doi.org/10.1038/s41598-024-78406-2 |
| work_keys_str_mv | AT georgemartvel automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT teddylazebnik automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT marcelofeighelstein automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT leahenze automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT sebastianmeller automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT ilanshimshoni automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT friederiketwele automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT alexandraschutter automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT noraforaita automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT sabinekastner automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT laurenfinka automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT stelioplluna automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT danielsmills automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT holgeravolk automatedvideobasedpainrecognitionincatsusingfaciallandmarks AT annazamansky automatedvideobasedpainrecognitionincatsusingfaciallandmarks |