Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learning
ABSTRACT: Poultry behavior indicates their health, welfare, and production performance. Timely access to broilers’ behavioral information can improve their welfare and reduce disease spread. Most behaviors require a period of observation before they can be accurately judged. However, the existing ap...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | Poultry Science |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S0032579124008939 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846123493403394048 |
|---|---|
| author | Yilei Hu Jiaqi Xiong Jinyang Xu Zhichao Gou Yibin Ying Jinming Pan Di Cui |
| author_facet | Yilei Hu Jiaqi Xiong Jinyang Xu Zhichao Gou Yibin Ying Jinming Pan Di Cui |
| author_sort | Yilei Hu |
| collection | DOAJ |
| description | ABSTRACT: Poultry behavior indicates their health, welfare, and production performance. Timely access to broilers’ behavioral information can improve their welfare and reduce disease spread. Most behaviors require a period of observation before they can be accurately judged. However, the existing approaches for multi-object behavior recognition were mostly developed based on a single-frame image and ignored the temporal features in videos, which led to misrecognition. This study proposed an end-to-end method for recognizing multiple simultaneous behavioral events of cage-free broilers in videos by Broiler Behavior Recognition System (BBRS) based on spatiotemporal feature learning. The BBRS consisted of 3 main components: the improved YOLOv8s detector, the Bytetrack tracker, and the 3D-ResNet50-TSAM model. The basic network YOLOv8s was improved with MPDIoU to identify multiple broilers in the same frame of videos. The Bytetrack tracker was used to track each identified broiler and acquire its image sequence of 32 continuous frames as input for the 3D-ResNet50-TSAM model. To accurately recognize behavior of each tracked broiler, the 3D-ResNet50-TSAM model integrated a temporal-spatial attention module for learning the spatiotemporal features from its image sequence and enhancing inference ability in the case of its image sequence less than 32 continuous frames due to its tracker ID switching. Each component of BBRS was trained and tested with the rearing density of 7 to 8 birds/m2. The results demonstrated that the mAP@0.5 of the improved YOLOv8s detector was 99.50%. The Bytetrack tracker achieved a mean MOTA of 93.89% at different levels of occlusion. The Accuracy, Precision, Recall, and F1score of the 3D-ResNet50-TSAM model were 97.84, 97.72, 97.65, and 97.68%, respectively. The BBRS showed satisfactory inference ability with an Accuracy of 93.98% when 26 continuous frames of the tracked broiler were received by the 3D-ResNet50-TSAM model. This study provides an efficient tool for automatically and accurately recognizing behaviors of cage-free multi-broilers in videos. The code will be released on GitHub (https://github.com/CoderYLH/BBRS) as soon as the study is published. |
| format | Article |
| id | doaj-art-8bca0d165e23400cbffdea756dd4ff4e |
| institution | Kabale University |
| issn | 0032-5791 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Poultry Science |
| spelling | doaj-art-8bca0d165e23400cbffdea756dd4ff4e2024-12-14T06:28:52ZengElsevierPoultry Science0032-57912024-12-0110312104314Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learningYilei Hu0Jiaqi Xiong1Jinyang Xu2Zhichao Gou3Yibin Ying4Jinming Pan5Di Cui6College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, P. R. China; Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province, Hangzhou 310058, P. R. ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, P. R. China; Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province, Hangzhou 310058, P. R. ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, P. R. China; Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province, Hangzhou 310058, P. R. ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, P. R. China; Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province, Hangzhou 310058, P. R. ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, P. R. China; Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province, Hangzhou 310058, P. R. ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, P. R. China; Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province, Hangzhou 310058, P. R. ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, P. R. China; Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province, Hangzhou 310058, P. R. China; Corresponding author.ABSTRACT: Poultry behavior indicates their health, welfare, and production performance. Timely access to broilers’ behavioral information can improve their welfare and reduce disease spread. Most behaviors require a period of observation before they can be accurately judged. However, the existing approaches for multi-object behavior recognition were mostly developed based on a single-frame image and ignored the temporal features in videos, which led to misrecognition. This study proposed an end-to-end method for recognizing multiple simultaneous behavioral events of cage-free broilers in videos by Broiler Behavior Recognition System (BBRS) based on spatiotemporal feature learning. The BBRS consisted of 3 main components: the improved YOLOv8s detector, the Bytetrack tracker, and the 3D-ResNet50-TSAM model. The basic network YOLOv8s was improved with MPDIoU to identify multiple broilers in the same frame of videos. The Bytetrack tracker was used to track each identified broiler and acquire its image sequence of 32 continuous frames as input for the 3D-ResNet50-TSAM model. To accurately recognize behavior of each tracked broiler, the 3D-ResNet50-TSAM model integrated a temporal-spatial attention module for learning the spatiotemporal features from its image sequence and enhancing inference ability in the case of its image sequence less than 32 continuous frames due to its tracker ID switching. Each component of BBRS was trained and tested with the rearing density of 7 to 8 birds/m2. The results demonstrated that the mAP@0.5 of the improved YOLOv8s detector was 99.50%. The Bytetrack tracker achieved a mean MOTA of 93.89% at different levels of occlusion. The Accuracy, Precision, Recall, and F1score of the 3D-ResNet50-TSAM model were 97.84, 97.72, 97.65, and 97.68%, respectively. The BBRS showed satisfactory inference ability with an Accuracy of 93.98% when 26 continuous frames of the tracked broiler were received by the 3D-ResNet50-TSAM model. This study provides an efficient tool for automatically and accurately recognizing behaviors of cage-free multi-broilers in videos. The code will be released on GitHub (https://github.com/CoderYLH/BBRS) as soon as the study is published.http://www.sciencedirect.com/science/article/pii/S0032579124008939broilerbehavior recognitionspatiotemporal featureend-to-endcomputer vision |
| spellingShingle | Yilei Hu Jiaqi Xiong Jinyang Xu Zhichao Gou Yibin Ying Jinming Pan Di Cui Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learning Poultry Science broiler behavior recognition spatiotemporal feature end-to-end computer vision |
| title | Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learning |
| title_full | Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learning |
| title_fullStr | Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learning |
| title_full_unstemmed | Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learning |
| title_short | Behavior recognition of cage-free multi-broilers based on spatiotemporal feature learning |
| title_sort | behavior recognition of cage free multi broilers based on spatiotemporal feature learning |
| topic | broiler behavior recognition spatiotemporal feature end-to-end computer vision |
| url | http://www.sciencedirect.com/science/article/pii/S0032579124008939 |
| work_keys_str_mv | AT yileihu behaviorrecognitionofcagefreemultibroilersbasedonspatiotemporalfeaturelearning AT jiaqixiong behaviorrecognitionofcagefreemultibroilersbasedonspatiotemporalfeaturelearning AT jinyangxu behaviorrecognitionofcagefreemultibroilersbasedonspatiotemporalfeaturelearning AT zhichaogou behaviorrecognitionofcagefreemultibroilersbasedonspatiotemporalfeaturelearning AT yibinying behaviorrecognitionofcagefreemultibroilersbasedonspatiotemporalfeaturelearning AT jinmingpan behaviorrecognitionofcagefreemultibroilersbasedonspatiotemporalfeaturelearning AT dicui behaviorrecognitionofcagefreemultibroilersbasedonspatiotemporalfeaturelearning |