Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap Images
Using public support to extract information from vast datasets has become a popular method for accurately labeling wildlife data in camera trap (CT) images. However, the increasing demand for volunteer effort lengthens the time interval between data collection and our ability to draw ecological infe...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Ubiquity Press
2024-12-01
|
| Series: | Citizen Science: Theory and Practice |
| Subjects: | |
| Online Access: | https://account.theoryandpractice.citizenscienceassociation.org/index.php/up-j-cstp/article/view/752 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841554911714607104 |
|---|---|
| author | Sarah E. Huebner Meredith S. Palmer Craig Packer |
| author_facet | Sarah E. Huebner Meredith S. Palmer Craig Packer |
| author_sort | Sarah E. Huebner |
| collection | DOAJ |
| description | Using public support to extract information from vast datasets has become a popular method for accurately labeling wildlife data in camera trap (CT) images. However, the increasing demand for volunteer effort lengthens the time interval between data collection and our ability to draw ecological inferences or perform data-driven conservation actions. Artificial intelligence (AI) approaches are currently highly effective for species detection (i.e., whether an image contains animals or not) and labeling common species; however, it performs poorly on species rarely captured in images and those that are highly visually similar to one another. To capitalize on the best of human and AI classifying methods, we developed an integrated CT data pipeline in which AI provides an initial pass on labeling images, but is supervised and validated by humans (i.e., a “human-in-the-loop” approach). To assess classification accuracy gains, we compare the precision of species labels produced by AI and HITL protocols to a “gold standard” (GS) dataset annotated by wildlife experts. The accuracy of the AI method was species-dependent and positively correlated with the number of training images. The combined efforts of HITL led to error rates of less than 10% for 73% of the dataset and lowered the error rates for an additional 23%. For two visually similar species, human input resulted in higher error rates than AI. While integrating humans in the loop increases classification times relative to AI alone, the gains in accuracy suggest that this method is highly valuable for high-volume CT surveys. |
| format | Article |
| id | doaj-art-06fcc329b5204ed2a3f5ce365465ab3e |
| institution | Kabale University |
| issn | 2057-4991 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Ubiquity Press |
| record_format | Article |
| series | Citizen Science: Theory and Practice |
| spelling | doaj-art-06fcc329b5204ed2a3f5ce365465ab3e2025-01-08T07:54:40ZengUbiquity PressCitizen Science: Theory and Practice2057-49912024-12-0191383810.5334/cstp.752734Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap ImagesSarah E. Huebner0https://orcid.org/0000-0001-5682-6467Meredith S. Palmer1https://orcid.org/0000-0002-1416-1732Craig Packer2https://orcid.org/0000-0002-3939-8162Smithsonian Conservation Biology InstitutePrinceton UniversityUniversity of MinnesotaUsing public support to extract information from vast datasets has become a popular method for accurately labeling wildlife data in camera trap (CT) images. However, the increasing demand for volunteer effort lengthens the time interval between data collection and our ability to draw ecological inferences or perform data-driven conservation actions. Artificial intelligence (AI) approaches are currently highly effective for species detection (i.e., whether an image contains animals or not) and labeling common species; however, it performs poorly on species rarely captured in images and those that are highly visually similar to one another. To capitalize on the best of human and AI classifying methods, we developed an integrated CT data pipeline in which AI provides an initial pass on labeling images, but is supervised and validated by humans (i.e., a “human-in-the-loop” approach). To assess classification accuracy gains, we compare the precision of species labels produced by AI and HITL protocols to a “gold standard” (GS) dataset annotated by wildlife experts. The accuracy of the AI method was species-dependent and positively correlated with the number of training images. The combined efforts of HITL led to error rates of less than 10% for 73% of the dataset and lowered the error rates for an additional 23%. For two visually similar species, human input resulted in higher error rates than AI. While integrating humans in the loop increases classification times relative to AI alone, the gains in accuracy suggest that this method is highly valuable for high-volume CT surveys.https://account.theoryandpractice.citizenscienceassociation.org/index.php/up-j-cstp/article/view/752ecologyconservationcamera trapssnapshot safarihuman-in-the-loopartificial intelligence |
| spellingShingle | Sarah E. Huebner Meredith S. Palmer Craig Packer Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap Images Citizen Science: Theory and Practice ecology conservation camera traps snapshot safari human-in-the-loop artificial intelligence |
| title | Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap Images |
| title_full | Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap Images |
| title_fullStr | Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap Images |
| title_full_unstemmed | Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap Images |
| title_short | Human Supervision is Key to Achieving Accurate AI-assisted Wildlife Identifications in Camera Trap Images |
| title_sort | human supervision is key to achieving accurate ai assisted wildlife identifications in camera trap images |
| topic | ecology conservation camera traps snapshot safari human-in-the-loop artificial intelligence |
| url | https://account.theoryandpractice.citizenscienceassociation.org/index.php/up-j-cstp/article/view/752 |
| work_keys_str_mv | AT sarahehuebner humansupervisioniskeytoachievingaccurateaiassistedwildlifeidentificationsincameratrapimages AT meredithspalmer humansupervisioniskeytoachievingaccurateaiassistedwildlifeidentificationsincameratrapimages AT craigpacker humansupervisioniskeytoachievingaccurateaiassistedwildlifeidentificationsincameratrapimages |