AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies
The proliferation of AI-powered bots and sophisticated fraudsters poses a significant threat to the integrity of scientific studies reliant on online surveys across diverse disciplines, including health, social, environmental and political sciences. We found a substantial decline in usable responses...
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-12-01
|
| Series: | Frontiers in Research Metrics and Analytics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frma.2024.1432774/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846145138517082112 |
|---|---|
| author | Natalia Pinzón Natalia Pinzón Vikram Koundinya Vikram Koundinya Ryan E. Galt Ryan E. Galt William O'R. Dowling Marcela Baukloh Namah C. Taku-Forchu Tracy Schohr Leslie M. Roche Leslie M. Roche Samuel Ikendi Mark Cooper Lauren E. Parker Lauren E. Parker Tapan B. Pathak Tapan B. Pathak |
| author_facet | Natalia Pinzón Natalia Pinzón Vikram Koundinya Vikram Koundinya Ryan E. Galt Ryan E. Galt William O'R. Dowling Marcela Baukloh Namah C. Taku-Forchu Tracy Schohr Leslie M. Roche Leslie M. Roche Samuel Ikendi Mark Cooper Lauren E. Parker Lauren E. Parker Tapan B. Pathak Tapan B. Pathak |
| author_sort | Natalia Pinzón |
| collection | DOAJ |
| description | The proliferation of AI-powered bots and sophisticated fraudsters poses a significant threat to the integrity of scientific studies reliant on online surveys across diverse disciplines, including health, social, environmental and political sciences. We found a substantial decline in usable responses from online surveys from 75 to 10% in recent years due to survey fraud. Monetary incentives attract sophisticated fraudsters capable of mimicking genuine open-ended responses and verifying information submitted months prior, showcasing the advanced capabilities of online survey fraud today. This study evaluates the efficacy of 31 fraud indicators and six ensembles using two agriculture surveys in California. To evaluate the performance of each indicator, we use predictive power and recall. Predictive power is a novel variation of precision introduced in this study, and both are simple metrics that allow for non-academic survey practitioners to replicate our methods. The best indicators included a novel email address score, MinFraud Risk Score, consecutive submissions, opting-out of incentives, improbable location. |
| format | Article |
| id | doaj-art-5f8bdfb882a84db7b92015733c29ffdd |
| institution | Kabale University |
| issn | 2504-0537 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Research Metrics and Analytics |
| spelling | doaj-art-5f8bdfb882a84db7b92015733c29ffdd2024-12-02T06:23:20ZengFrontiers Media S.A.Frontiers in Research Metrics and Analytics2504-05372024-12-01910.3389/frma.2024.14327741432774AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategiesNatalia Pinzón0Natalia Pinzón1Vikram Koundinya2Vikram Koundinya3Ryan E. Galt4Ryan E. Galt5William O'R. Dowling6Marcela Baukloh7Namah C. Taku-Forchu8Tracy Schohr9Leslie M. Roche10Leslie M. Roche11Samuel Ikendi12Mark Cooper13Lauren E. Parker14Lauren E. Parker15Tapan B. Pathak16Tapan B. Pathak17Geography Graduate Group, University of California, Davis, Davis, CA, United StatesRhizobia LLC, San Francisco, CA, United StatesGeography Graduate Group, University of California, Davis, Davis, CA, United StatesUniversity of California Cooperative Extension, Davis, CA, United StatesGeography Graduate Group, University of California, Davis, Davis, CA, United StatesAgricultural Sustainability Institute, College of Agricultural and Environmental Sciences, University of California, Davis, Davis, CA, United StatesRhizobia LLC, San Francisco, CA, United StatesRhizobia LLC, San Francisco, CA, United StatesNatural Resources Institute, University of Wisconsin-Madison, Madison, WI, United StatesUniversity of California Cooperative Extension, Davis, CA, United StatesGeography Graduate Group, University of California, Davis, Davis, CA, United StatesUniversity of California Cooperative Extension, Davis, CA, United StatesDivision of Agriculture and Natural Resources, University of California, Merced, Merced, CA, United StatesGeography Graduate Group, University of California, Davis, Davis, CA, United StatesGeography Graduate Group, University of California, Davis, Davis, CA, United StatesCalifornia Climate Hub, United States Department of Agriculture (USDA), Davis, CA, United StatesUniversity of California Cooperative Extension, Davis, CA, United StatesDivision of Agriculture and Natural Resources, University of California, Merced, Merced, CA, United StatesThe proliferation of AI-powered bots and sophisticated fraudsters poses a significant threat to the integrity of scientific studies reliant on online surveys across diverse disciplines, including health, social, environmental and political sciences. We found a substantial decline in usable responses from online surveys from 75 to 10% in recent years due to survey fraud. Monetary incentives attract sophisticated fraudsters capable of mimicking genuine open-ended responses and verifying information submitted months prior, showcasing the advanced capabilities of online survey fraud today. This study evaluates the efficacy of 31 fraud indicators and six ensembles using two agriculture surveys in California. To evaluate the performance of each indicator, we use predictive power and recall. Predictive power is a novel variation of precision introduced in this study, and both are simple metrics that allow for non-academic survey practitioners to replicate our methods. The best indicators included a novel email address score, MinFraud Risk Score, consecutive submissions, opting-out of incentives, improbable location.https://www.frontiersin.org/articles/10.3389/frma.2024.1432774/fullsurveysonline data collectionfraud detectionsurvey farmsAI bots |
| spellingShingle | Natalia Pinzón Natalia Pinzón Vikram Koundinya Vikram Koundinya Ryan E. Galt Ryan E. Galt William O'R. Dowling Marcela Baukloh Namah C. Taku-Forchu Tracy Schohr Leslie M. Roche Leslie M. Roche Samuel Ikendi Mark Cooper Lauren E. Parker Lauren E. Parker Tapan B. Pathak Tapan B. Pathak AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies Frontiers in Research Metrics and Analytics surveys online data collection fraud detection survey farms AI bots |
| title | AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies |
| title_full | AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies |
| title_fullStr | AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies |
| title_full_unstemmed | AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies |
| title_short | AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies |
| title_sort | ai powered fraud and the erosion of online survey integrity an analysis of 31 fraud detection strategies |
| topic | surveys online data collection fraud detection survey farms AI bots |
| url | https://www.frontiersin.org/articles/10.3389/frma.2024.1432774/full |
| work_keys_str_mv | AT nataliapinzon aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT nataliapinzon aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT vikramkoundinya aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT vikramkoundinya aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT ryanegalt aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT ryanegalt aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT williamordowling aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT marcelabaukloh aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT namahctakuforchu aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT tracyschohr aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT lesliemroche aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT lesliemroche aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT samuelikendi aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT markcooper aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT laureneparker aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT laureneparker aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT tapanbpathak aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies AT tapanbpathak aipoweredfraudandtheerosionofonlinesurveyintegrityananalysisof31frauddetectionstrategies |